Writing an IFR Assembler for UEFI - Part 4

This is part 4 of Writing an IFR Assembler for UEFI.

To set up the IFR Assembler, follow the instructions in Part 1.
To learn about the higher-level parsing process of the IFR Assembler, view Part 2.
To learn about the token-parsing process of the IFR Assembler, view Part 3.

This post will walk you through the expression-parsing process of the IFR Assembler.

Understanding Expression Hierarchy

Each expression is a series of operators and operands. Operators are actions that can be performed on values. These actions include addition and subtraction, multiplication and division, and comparisons along with many other actions. Operands are all of the values that the operators act upon.

In any mathematical equation, there is an order of precedence. Order of precedence is the order in which the different operators in an equation are executed. This is why multiplication is done before addition in every equation. Expressions in C, like mathematic equations, have their own order of precedence to organize the order in which the operators are processed.

To see the complete order of precedence for the C programming language, click here.

The Expression-Parsing Process

Part 2 of this series discussed ParseOperand(), which calls a different parsing function depending on what type of operand is found.











If an expression is found, ParseExprOperand() is called, and ParseExprOperand() calls ParseExpr().











ParseExpr() is the beginning of a large recursive loop. Before anything is even parsed, ParseExpr()
calls ParseExpr2(), and ParseExpr2() calls ParseExpr3(), and this process goes on until ParseExpr12().














The functions ParseExpr() through ParseExpr12() determine the order of precedence for the expression parsing. ParseExpr12() hold the operands and operators with the highest precedence, and ParseExpr() holds those with the lowest. These functions tell the parser what to do with the each operator and operand and what type of token to expect next. See table 4.1 for a full list of operands and operators parsed by each function.

 

 

Table 4.1 - Operands and Operators Parsed by Each Function



 

When operators within an expression are parsed, they are turned into their own sub-expression. Depending on the number of operands that the operator requires, a different function is called.  For example, if the operator has only one operand, like ++ (T_P__PLUS_PLUS), it is parsed with the function CreateExprUnary().





On the other hand, if the operator is like + (T_P__PLUS), which has two operands, the function CreateExprBinary() is called.












If the parser finds a question mark (T_P__QUESTION), that means that there is a conditional statement involved (EXPR_O_CONDITIONAL). Conditional statements are the only trinary expressions, which requires three operands, so if a question mark is found, CreateExprTrinary() is called. These functions create expressions for the operators and expect a specific number of operands.









If the operands in the expression are unsigned integers (T_P_UINT), CreateExprUint() is called. This function creates an expression from that integer.











If the operand is either true (T_P_TRUE) or false (T_P_FALSE), the function CreateExprBoolean() is called to make the expression with a Boolean response.











If the operand found turns out to be an IFR opcode, it is treated like a function and function parameters are added to the operand. These operands are found within the parentheses of the opcode and are not written as a separate expression. Instead, their information is recorded as operands of the opcode's expression. A different function is called depending on the number of operands the opcode can hold. If the opcode has a set number of operands, then CreateFn() is called.










 If the number of operands within an opcode is variable, then CreateVarFn() is called. In each of these function, all the operands of the opcode are written into the expression with the opcode itself.















 If found within an expression, a left parenthesis (T_P_LPAREN) represents the beginning of another expression nested within the first. For example, in the expression  x * (y - z), the section y - z can stand alone as its own expression since it is inside parentheses. Because the  expression within the parentheses is, in fact, a separate expression, ParseExpr() is called recursively to parse the nested expression until it finds the expression's end, which is marked by a right parenthesis (T_P_RPAREN). After being parsed, the entire nested expression is returned and is treated as a single operand. In our example, the (y - z) would be treated as a single operand, w, so that the equation would read x * w.

 

 

 

 

 

The Expression Tree

Once the parser begins going through the expression, it begins to organizes the operands and operators by order of precedence. The way they are organized can be described as a type of expression tree. The higher the precedence of a specific token, the closer it will be to the leaves of the expression tree. Inversely, the lower the precedence of a token, the closer the token will be to the roots of the tree. In the expression a * b + c + d, the expression-parser would begin parsing the expression. First, the parser would point to a. Since a is an operand and operands receive the highest precedence, a will be set as a leaf in the tree.











The parser would then point to the asterisk, which serves as the multiplication operator. Since the asterisk is an operator, it has a lower precedence than a, so the multiplication operator is moved up a level. The multiplication operator requires two operands, each one represented by a branch coming off of it in the tree. a is set to one of operands of the multiplication operator.













When b is reached, the parser sees that the multiplication operator still has another required operand, so b is set as that operand.



















The parser then moves to the first plus sign, which is the addition operator. Since it is not an operand, the plus sign is moved up a level. Since the plus sign has less precedence than the asterisk, it is moved a level higher than the asterisk. The asterisk is set as one of the operands of the plus sign. The next token, c, is set as the other operand for the plus sign.













The parser moves on to the second plus sign. Since the plus sign has less precedence then an operand, the plus sign moves up a level from c. Though both plus signs are on the same level of precedence, precedence goes left to right for plus signs, so the first plus sign takes precedence. The second plus moves a level higher than the first plus sign, becoming the root of the expression tree. The parser then moves onto d, which becomes the second operand of the second plus sign.

 

 

 

 

Writing Expressions

All expression are put within an opcode, usually a conditional. When an expression is finished being parsed, it is returned to ParseForm(), which writes the opcode's information into the output buffer. In order to write the opcode's expression to the output buffer, the function WriteExpr() is called.









WriteExpr() calls WriteExprHelper(), which then takes the expression and creates an opcode out of the expression and its operands.  After creating the opcode, the opcode is returned to WriteExpr(), which writes the opcode to the output buffer.

Normally, operands are written to the buffer before the operator is written, so if a + b is found, the opcode, a b + would be written to the output buffer. If an expression does has a specific order for its operators and operands to be written in, an expression-specific function is called. See Table 4.2 to view these expression-specific functions.

Table 4.2 - The table below shows expression-specific expression-writing functions called by WriteExprHelper().

This finishes our series on Writing the IFR Assembler for UEFI. The IFR Assembler used does not several situation. The IFR Assembler does not handle any packages other than form packages nor does it have an optimization process. Also, the IFR Assembler is not free from bugs. This Assembler was simply made to as an example of how to set up an IFR Assembler for UEFI.

 

Table 4.3 - Operands and Operators Parsed by ParseExpr12() 

 

 

Writing an IFR Assembler for UEFI - Part 3

This is Part 3 of Writing an IFR Assembler for UEFI.

This article discusses the IFR Assembler set up in Part 1. To set up the IFR Assembler, follow the instructions in Part 1.
To learn about the higher-level parsing process of the IFR Assembler, view Part 2.

This post will walk you through the token-parsing process of the IFR Assembler.

Understanding Token Parsing

The last article talked about the higher-level parsing system and the function that the IFR Assembler uses to parse each item. Token-parsing is the second part of the parsing system which goes through each line of code in the .pl file and parses each significant syntactic element.

A significant syntactic element is any part of the .pl file that is part of the actual code. This includes op-codes, punctuation, expressions, and many other parts of the code. Items of code that would not be considered significant syntactic element include comments and white space. The parser reads these significant syntactic elements, or tokens, on behalf of the higher-level parsing functions.

The Parsing Process

The first part of token-parsing is finding the inputs to the parser. There are three main variables that must be defined. The first two variables are SourceFileName and SourceFileLine, both of which are used to give an error's location when an error is printed. The final input of token-parsing is the variable psz. psz points to a character within the code, usually the first character in a token, and is used to find out what token is currently being pointed to.



These three inputs are used mainly in the function tokenP(), which is within the source file Parse.c. tokenP() reads what character psz is pointing to, and uses the characters to figure out which token psz is currently pointing to. After the type of token is discovered, tokenP() sets the variable t to equal that token. After t is set, psz is changed to point to the first character of the next token in the series.













Because t is a numeric value, every token has a numeric value connected to it so that t can represent that token. Each of these values is defined in the source file Token.h.

Each time that tokenP() is called, it will return a token in the form of the variable t. These tokens are passed to the high-parsing functions discussed in Part 2, which check the syntax of the code.










If the token is a GUID (T_P_UUID), an unsigned integer (T_P_UINT), an ASCII string (T_P_STRA), the value of the token is put into tu, tguid, tstrA, respectively. These variables are then returned to the higher-level parsing functions along with t. The functions InitParse() and ShutParse() help to set up and empty out these variables, namely tstrA and tstrW, before and after the parsing of a token.

tokenP() calls tokenNL() when psz reaches the end of a line of code. tokenNL() points psz to the next line of code and increments SourceFileLine.



 

The function backslash() allows strings to use escape sequences.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The utility function dispP() prints out a different string for each token value in Token.h. dispP() is mainly used to write out the name of the token when an error is printed.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

parseguid() is used to parse GUIDs that are written in the following formats: 

{ 0x49adf016, 0x4177, 0x48ae, { 0xb2, 0x55, 0x92, 0x90, 0xa8, 0x20, 0x5c, 0x9d } }
 

or
 

 { 0x49adf016, 0x4177, 0x48ae, 0xb2, 0x55, 0x92, 0x90, 0xa8, 0x20, 0x5c, 0x9d }

 

 

 

 

 

 

 

  

 

The table below describes what each function token-specific parsing function does.

 

 The token-parser breaks down a series of tokens so that the higher-level parsing functions can check the syntax of the code. The next article will teach you how expressions are parsed and written.

 



Writing an IFR Assembler for UEFI - Part 2

This is part two of Writing the IFR Assembler for UEFI. This article discusses the IFR Assembler set up in Part 1. To set up the IFR Assembler, follow the instructions in Part 1.

This blog post will walk you through the basic parsing process of the IFR Assembler from the parsing of the Command Line to the parsing of individual forms.

Understanding the Parsing System

Normally, the IFR Assembler is given a .pl file from the command line to parse. The .pl file holds the information about the packages, form sets, forms, and everything that needs to be parsed by the Assembler. There are two types of places to put information for each of these items to be parsed. First, information that relates specifically to that item, like operands of a form, are put within parentheses right after the item. If there is information nested inside an item, like a form nested within a form set, the information is put into the curly braces after.

When parsing a .pl file, the IFR Assembler starts with the first item, which is usually the Package List, the item is parsed, then anything nested within is operated, which would be the Form Package. After the Form Package is parsed, everything nested in front of the Form Package is parsed, and this cycle goes on until the Assembler has parsed everything. When it has, the Assembler writes out the parsed information to the output file. The following two sections will serve as a basic walk through of how text on a command line is parsed and written to the output file.

Command Line Parsing

The IFR Assembler application is entered through the function main() in the program IfrAsm.c. One of the important jobs of main() is to translate information given to it from the command line so that it can be parsed by the IFR Assembler.












To begin parsing the command line, InitCmdLine() is called to initialize command line global variables.







After the initialization is finished, main() calls ParseCmdLine(). ParseCmdLine() and ParseCmdLineOption() go through the command line and create an array of the actions and locations written into the command line.










main() uses this array to get information from the locations. This information is then made into a source file.

Parsing Within Packages

The source file is sent to the function ConvertSourceFileToPackages(), which is within ParsePackage.c. This function, like the name suggests, converts the source files into packages. ConvertSourceFileToPackages() puts all the packages in to a package list which is sent down through ParsePackage.c to get parsed.

The package list is sent to ParsePackage(). This parses the contents of the packages and depending on the type of package found, a different parsing function is called. Currently, only form packages are recognized by the Assembler, and if the packages found are not form packages, an error is produced.

Form packages are sent to ParseFormPackages(). This function takes the form package and goes throught its contents. Normally, a Form Package carries Form Sets. Each set is sent to ParseForms() to be parsed further.










The contents of the Form Package are sent to ParseForms() which goes through the contents and parses each form set within the package.

ParseForm() is given a single form from ParseForms(), and it reads the op-codes that are held within the form. When ParseForm() reaches an opcode, it checks to see if the op-code can have any operands (This information on op-codes is found in Opcodes[] in Parse Package. If the op-code does, ParseForm() checks for a left parenthesis). If the operand requires an operand, the left parenthesis is required, otherwise, the left parenthesis is optional. If a left parenthesis is found, ParseForm() sends the operands within the parentheses to ParseOperand().










ParseOperand() decides what type of operand it was passed and tests to see if the operand type found matches the operand type expected. If the operand type found matches what is expected, ParseOperand()  decides which function to send the operand to. There are specific functions for each type of operand, like ParseUint8() or ParseExprOperand(). Each operand-specific function has a unique set of tasks that it performs in order to parse the operand properly.








After the operands are all parsed, the form's information is returned back to ParseForm() in a buffer to be written to the output file. . When all the forms, lists, and packages have been parsed and written to the output file, the program returns back to main(). main() frees information and clears out any remaining information.

The table below gives the basic information of each parsing function discussed in this article, including its name, what item it parses, its location in the assembler, and which section of the UEFI Specification it refers.

 

 

 

When a test case like the one used in Part 1 is entered into the command line, this chain of parsing functions is what takes the commands on the command line, translates it, and writes it to the output file. In the next article, you will learn about token parsing in the IFR Assembler.

Writing an IFR Assembler for UEFI - Part 1

In this article, you will learn how to run SysLib and IFR Assembler in Visual Studio 2012.
This article assumes that you have Visual Studio 2012 and that you have built EDK2's NT32 platform in C:\sourcecode\edk2.

 

In order to make these steps work as expected, you must set up Visual Studios as instructed in the blog post here.

 

Set Up:

 



 

1. To find the SysLib code, you must use Subversion to download the files onto the computer using this link: https://svn.code.sf.net/p/syslibforuefi/code/trunk
 

2. Create a folder titled SysLib, then take all the downloaded files and put them into the SysLib folder. Keep the files organized in folders in the same way they are on the website.

 

 

 

 

3. Copy the SysLib folder you just created into the folder C:\sourcecode\edk2.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

4. Open the folder C:\sourcecode\edk2\Nt32Pkg

 

 

 

 

 

 

 

 

 

 

 

 

 

 

5. Open Nt32Pkg.dsc in Visual Studio.

 

 

 

 

 

 

 

 

 

 

6. On about line 109 of Nt32Pkg.dsc, which is under the section "Generic Modules", write the following code:

SysLib|SysLib/Source/SysLib.inf

 

 

 

 

 

 

 

 

7. On about line 384 of Nt32Pkg.dsc, which is under the section "DXE Phases modules", write the following code: SysLib/Applications/IfrAsm/IfrAsm.inf

 

 

 

 

 

 

 

8. While downloading the files for SysLib, you should have downloaded some test cases. They are held in the folder C:\sourcecode\edk2\SysLib\Application\IfrAsm. Each one is a .pl file and is labeled 1 through 10. Select all the test cases.

 

 

 

9. Right click on the selected test cases and click on "Cut".

 

 

 

 

 

 

 

 

 

 

 

10. Open the folder C:\sourcecode\edk2\Build\NT32\DEBUG_VS2010x86\IA32.

 

 

 

 

 

 

 

 

 

 

 

11. Right click and select "Paste" to add all test cases to the folder.

 

 

 

 

 

 

 

 

 

 

 

 

 Running

12. Open Visual Studio 2012. Make sure that by this point, you have created EDK2's NT32 environment. If not, go set it up using these instructions.

13. Open your NT32 project.
















14. Click on Build->Build Solution or press F7.






15. Wait for the program to finish building.







16. Once the build has finished, click on Debug->Start Debugging or press F5.











17. A window will appear, press the button that says "No".
















18. Two or three windows will appear. Go to one of the GOP windows. Press "Enter" until Shell> _  appears at the bottom of the window.











19. Type f8: into the GOP window, then press "Enter".













20. When f8:\> _ appears at the bottom of the GOP window, enter the following text:
IfrAsm testcase8.pl -o testcase8.ifr











21. The opcodes used for testcase8.pl will be displayed in the GOP window. To run any of the other test cases, simply type in the text you typed in step 21, but replace testcase8 with the name of the test case you would like to run.










Test cases like the one used in the blog post are useful because the can be used to test different parts of the code to ensure that the code is being correctly parsed, read, and written. In the next post, the basic parsing organization of an IFR Assembler will be explained.