HOW TO: Disassembling the HII Database (Part 5)

During this series, we've been looking at how to disassemble the contents of the UEFI HII database, which is described in chapter 28 of the UEFI specification. So far, we've managed to take apart the package lists and break those into the different types of packages. The Forms type of package are encoded using Internal Forms Representation (IFR), a binary format made up of opcodes. Each opcode describes an object and opcodes can be nested inside of other opcodes or, as we say in HII, they are child objects in the parent object's scope. In part 4, we looked at the first sets of opcodes: Form Sets, which are collections of forms, and the 3 different types of variable stores (buffer, name/value and EFI variable), which describe how configuration settings map back to non-volatile storage.

This time we'll take a look at two more object types (Default Stores and Forms) that can be direct children of Form Sets.

Default Stores

Default Stores are collections of default configuration setting (Question) values, grouped by purpose. There are four different "standard" types of default stores

1. Normal - The normal set of defaults.
2. Manufacturing - The defaults to use during manufacturing.
3. Safe - The defaults to use to make the system as stable as possible.
4. OEM - There is a large range of default store types set aside for special use by OEM forms.

The default values can be loaded using the RESET_BUTTON Statement or through a browser-specific mechanism.

There are four ways to assign a default value to a specific default store. These are listed from lowest priority to highest priority:

1. CHECKBOX Question Flag. For this specific type of Question, a default value of On or Off can be assigned to either the Normal or Manufacturing default store type.
2. Question's Child ONE_OF_OPTION. Question's can have child objects that allow a user-readable name to be assigned to a specific value. Another set of flags also specifies whether the value should serve as the Normal or Manufacturing default store type for the parent Question.
3. DEFAULT Object. Question's can have child objects that allow a value be assigned to a any default store type.
4. Question Callback. Question's can provide a callback function that optionally provides a value for any default store type.

EFI_IFR_DEFAULT_STORE

The actual IFR encoding for the structure looks like this:
 typedef struct _EFI_IFR_DEFAULTSTORE {   EFI_IFR_OP_HEADER Header;  EFI_STRING_ID DefaultName;   UINT16 DefaultId; } EFI_IFR_DEFAULTSTORE; The Header  is the normal IFR opcode header. DEFAULT_STORE opcodes never have any child objects, so the Scope bit is never set. The DefaultId identifies the default store type (0 = Normal, 1 = Manufacturing, 2 = Safe and other platform-specific types are 0x4000+). The DefaultName gives the string identifier for a string that gives a user-readable name for the default store. While a platform might already have a name for 0,1 and 2, it is unlikely it would have one for any others. 

UEFI_HII_DEFAULT_STORE

This IFR opcode gets translated into the default store container: typedef struct _UEFI_HII_DEFAULT_STORE_P {

  SYS_OBJ Obj;

 

 

  EFI_STRING_ID Name;                   // default store name.

  UINT16 Id;                            // default store id.

 

 

} UEFI_HII_DEFAULT_STORE_P;
typedef UEFI_HII_DEFAULT_STORE_P UefiHiiDefaultStoreP;

UefiHiiParseFormDefaultStoreOp()

Pretty straight forward. Now here's the parsing code, which checks for input errors. Then it verifies that the default store has not already been declared for this form set. DefaultIds are global across all forms in a form set, so it is possible that they are duplicates. If no duplicate, a new Default Store container is created, initialized and added to the current Form Set.

EFI_STATUS

UefiHiiParseFormDefaultStoreOp (

  IN EFI_IFR_OP_HEADER *Op,

  IN OUT UEFI_HII_FORM_PKG_STATE *S

  )

{

  EFI_STATUS s;

  EFI_IFR_DEFAULTSTORE *DefaultStoreOp;

  UEFI_HII_DEFAULT_STORE_P *DefaultStoreP;

  SYS_LIST_POS pos;

 

 

  DefaultStoreOp = (EFI_IFR_DEFAULTSTORE *) Op;

  if (!UefiHiiOpIsValid (Op, S) ||

      !UefiHiiOpInFormSet (Op, S) ||

      !UefiHiiOpNotInForm (Op, S)) {

    return EFI_INVALID_PARAMETER;

  }

 

 

  ...dump out debug information...

  for (pos = NULL;
       SysListGetNext (&S->FormSetP->DefaultStores, &pos, &DefaultStoreP); ) {

 

 

 

    if (!SysIsValid (DefaultStoreP)) {

      continue;

    }

 

 

 

    if (DefaultStoreP->Id == DefaultStoreOp->DefaultId) {

      return EFI_SUCCESS;

    }

  }

 

 

  DefaultStoreP = SysNew (UefiHiiDefaultStoreP);

  if (DefaultStoreP == NULL) {

    s = EFI_OUT_OF_RESOURCES;

    goto exit;

  }

 

 

  if (!SysListAddTail (&S->FormSetP->DefaultStores, DefaultStoreP)) {

    s = EFI_OUT_OF_RESOURCES;

    goto exit;

  }

 

 

  s = EFI_SUCCESS;

exit:

  return s;

}

 

Forms

With Forms, we really get into the heart of the HII parsing. Each Form has the following attributes:

1. Form Identifier. Uniquely identifies this form from all others in the same Form Set.
2. Title String. String to associate with this Form.
3. Image. Optional image to assocate with this Form. Specified by a nested IMAGE opcode.
4. Animation. Optional animation to associate with this Form. Specified by a nested ANIMATION opcode.
5. List of Rules. These are essentially shortcuts or optimizations or "functions" that can be used by any expression in the Form. Each Rule is associated with a Rule Identiifer. Specified by a nested RULE opcode.
6. List of Statements. These are either static display items (pure Statements) or display items with configurable values (Questions). There are a dozen or so Statement and Question opcodes.
7. Modal Flag. Indicates whether the user can navigate easily away from this Form (FALSE, using menus) or else must click a button (TRUE). Specified by a nested MODAL_TAG opcode.
8. Locked. Indicates that the Form and its contents are protected from binary editing. Specified by a nested LOCKED opcode.
9. Methods. A method indicates a style by which a specific form should be interacted with. This is designed to support the way in the configuration settings controlled by a form would interact with an industry standard or OEM specification. There is only one method described in the UEFI specification (the standard method) and this is the method type assigned to all forms introduced with the FORM opcode. To create any other form types, the FORM_MAP opcode should be used.

EFI_IFR_FORM, EFI_IFR_FORM_MAP

Here are the two IFR opcode definitions:

typedef struct _EFI_IFR_FORM {
  EFI_IFR_OP_HEADER Header;
 
  EFI_FORM_ID FormId;
  EFI_STRING_ID FormTitle;
} EFI_IFR_FORM; 

typedef struct _EFI_IFR_FORM_MAP_METHOD {
  EFI_STRING_ID MethodTitle;
  EFI_GUID MethodIdentifier;
} EFI_IFR_FORM_MAP_METHOD;

typedef struct _EFI_IFR_FORM_MAP {
  EFI_IFR_OP_HEADER Header;
  EFI_FORM_ID FormId;
//EFI_IFR_FORM_MAP_METHOD Methods[];
} EFI_IFR_FORM_MAP; 

 

The FORM_MAP opcode is variable in length, holding up to three Methods names and GUIDs. The FORM opcode is of fixed length.

UEFI_HII_FORM_P

The application also has a Form container:

 

typedef struct _UEFI_HII_FORM_P {
  SYS_OBJ Obj;

 

  EFI_FORM_ID Id;

  EFI_STRING_ID Title;
  EFI_IMAGE_ID Image;
  EFI_ANIMATION_ID Animation;

 

  BOOLEAN Locked;
  BOOLEAN Modal;                     

 

  SYS_ARRAY MethodIds;
  SYS_ARRAY MethodTitles;

 

  SYS_MAP_U16P Rules;                   // rule id->expression

  SYS_LIST_O Stmts;                     // statements on form, in order.
} UEFI_HII_FORM_P;

 

typedef UEFI_HII_FORM_P UefiHiiFormP;

The Form container contains fields for all of the various attributes. Some of the attributes are optional and only filled further if a nested child opcode contains additional detail.

 

UefiHiiParseFormFormOp(), UefiHIiParseFormFormMapOp()

The code for the FORM and FORM_MAP opcodes is very similar. Both check the input parameters, create a Form container, fill in the Method type and title and add the Form to the Form Set. Also, each changes the current Form Package parsing state so that the current Form is this Form.

 

The Form Map has a small amount of additional code to handle the optional extra Method information, while Form sets a default Method. In this way, the two opcodes are handled identically.

 

EFI_STATUS
UefiHiiParseFormFormOp (
  IN CONST EFI_IFR_OP_HEADER *Op,
  IN OUT UEFI_HII_FORM_PKG_STATE *S
  )
{
  EFI_STATUS s;
  EFI_IFR_FORM *FormOp;
  UINT16 MethodTitle;

  MethodTitle = 0;

  FormOp = (EFI_IFR_FORM *) Op;

 

  if (!UefiHiiOpIsValid (Op, S) ||
      !UefiHiiOpInFormSet (Op, S) ||
      !UefiHiiOpNotInForm (Op, S)) {
    return EFI_INVALID_PARAMETER;
  }

 

  ...dump out form information...

 

  S->FormP = SysNew (UefiHiiFormP);
  if (S->FormP == NULL) {
    s = EFI_OUT_OF_RESOURCES;
    goto exit;
  }

  S->FormP->Id = FormOp->FormId;
  S->FormP->Title = FormOp->FormTitle;

 

  SysArrayAppend (&S->FormP->MethodIds, &gEfiHiiStandardFormGuid);
  SysArrayAppend (&S->FormP->MethodTitles, &MethodTitle);

  if (!SysListAddTail (&S->FormSetP->Forms, S->FormP)) {
    SysDelete (S->FormP);
    S->FormP = NULL;
    s = EFI_OUT_OF_RESOURCES;
    goto exit;
  }

  s = EFI_SUCCESS;
exit:
  return s;
}
  

EFI_STATUS
UefiHiiParseFormFormMapOp (
  IN CONST EFI_IFR_OP_HEADER *Op,
  IN OUT UEFI_HII_FORM_PKG_STATE *S
  )
{
  UINT32 i;
  EFI_STATUS s;
  EFI_IFR_FORM_MAP *FormMapOp;
  EFI_IFR_FORM_MAP_METHOD *FormMapMethod;
  UINT32 FormMapMethodCount;

  FormMapOp = (EFI_IFR_FORM_MAP *) Op;

  if (!UefiHiiOpIsValid (Op, S) ||
      !UefiHiiOpInFormSet (Op, S) ||
      !UefiHiiOpNotInForm (Op, S)) {
    return EFI_INVALID_PARAMETER;
  }

 

  FormMapMethod = (EFI_IFR_FORM_MAP_METHOD *)(FormMapOp + 1);
  FormMapMethodCount =

    (Op->Length - sizeof (EFI_IFR_FORM_MAP))/sizeof(EFI_IFR_FORM_MAP_METHOD);
  if (((FormMapMethodCount * sizeof (EFI_IFR_FORM_MAP_METHOD)) +

        sizeof (EFI_IFR_FORM_MAP)) != Op->Length) {
    return EFI_INVALID_PARAMETER;
  }
  if (FormMapMethodCount == 0) {
    return EFI_INVALID_PARAMETER;
  }

 

  ... dump out form map opcode information...

 

  S->FormP = SysNew (UefiHiiFormP);
  if (S->FormP == NULL) {
    SYSINFO ("out of memory.\n");
    s = EFI_OUT_OF_RESOURCES;
    goto exit;
  }

  S->FormP->Id = FormMapOp->FormId;

 

  for (i = 0; i < FormMapMethodCount; i++) {
    SysArrayAppend (&S->FormP->MethodTitles, &FormMapMethod[i].MethodTitle);
    SysArrayAppend (&S->FormP->MethodIds, &FormMapMethod[i].MethodIdentifier);
  }

  if (!SysListAddTail (&S->FormSetP->Forms, S->FormP)) {
    SysDelete(S->FormP);
    S->FormP = NULL;
    s = EFI_OUT_OF_RESOURCES;
    goto exit;
  }

  s = EFI_SUCCESS;
exit:
  SysEmpty (&Guid);
  return s;
}

UefiHiiParseFormFormEndOp()

This function is closed at the end of the FORM or FORM_MAP opcode's scope. All it does is update the Form Package parsing state so that there is no current Form.

EFI_STATUS
UefiHiiParseFormFormEndOp (
  IN CONST EFI_IFR_OP_HEADER *Op,
  IN OUT UEFI_HII_FORM_PKG_STATE *S
  )
{
  if (!UefiHiiOpIsValid (Op, S)) {
    return EFI_INVALID_PARAMETER;
  }
  S->FormP = NULL;
  return EFI_SUCCESS;
}

Conclusion

Now we've made it down to the Form level of the IFR parsing. Next, we will take a look at the Statement and the parsing and storage for the three pure statements (Text, Subtitle and Reset Button).

Until next time, keep digging deeper.

HOW TO: Disassembling the HII Database (Part 4)

The HII Database is the portion of the UEFI Specification that manages the user-interface resources, like forms, fonts, images and strings. The tool we've been developing parses the contents of the database and displays it. Up to this point, we've broken down the data into the constituent package lists, and then broken down package lists into packages, and then started looking at how form packages are encoded.

Form Set 

Form packages are encoded as a series of variable-length data structures, called opcodes. The previous article shows how the opcodes are parsed, and then a function for when the opcode is first encountered, and then another function for when the opcode's scope is completed.

EFI_IFR_FORM_SET

The top level objects in a forms package are all form sets. The form set describes a collection of forms, variable stores and default stores, associated with a GUID, prompt text, help text, an image and an animation.

typedef struct _EFI_IFR_FORM_SET {
  EFI_IFR_OP_HEADER        Header;
  EFI_GUID                 Guid;
  EFI_STRING_ID            FormSetTitle;
  EFI_STRING_ID            Help;
  UINT8                    Flags;
//EFI_GUID                 ClassGuid[];
} EFI_IFR_FORM_SET;

The Header  the standard IFR opcode header. The Guid uniquely identifies the form set. The title of the form set (FormSetTitle) and there is also help text (Help). There can be up to three class GUIDs, which provide a way to classify the form sets. These GUIDs (along with Guid) can be used in the SendForm() function call to specify which types of form sets will appear in the user-interface.

UefiHiiParseFormFormSetOp()

The top level objects in a forms package are all form sets. Form sets are collections of forms associated with a GUID.

EFI_STATUS
UefiHiiParseFormFormSetOp (
  IN CONST EFI_IFR_OP_HEADER *Op,
  IN OUT UEFI_HII_FORM_PKG_STATE *S
  )
{
  UINT8 ClassGuidCount;
  EFI_GUID *ClassGuid;
  EFI_STATUS s;
  EFI_IFR_FORM_SET *FormSetOp;
  SYS_STRA Guid;
  UINT32 i;

  FormSetOp = (EFI_IFR_FORM_SET *) Op;

  if (!UefiHiiOpIsValid (Op, S)) {
    return EFI_INVALID_PARAMETER;
  }
  if (S->FormSetP != NULL) {
    SYSINFO ("FORM_SET: Cannot be inside a FORM_SET opcode.\n");
    return EFI_INVALID_PARAMETER;
  }

  ClassGuid = (EFI_GUID *)(FormSetOp + 1);
  ClassGuidCount = FormSetOp->Flags & 0x03; // isolate class GUID count.
  if (ClassGuidCount * sizeof (EFI_GUID) + sizeof (EFI_IFR_FORM_SET) >
      Op->Length) {
    ...
    return EFI_INVALID_PARAMETER;
  }

  ...dump out debug information...

  S->FormSetP = SysNew (UefiHiiFormSetP);
  if (S->FormSetP == NULL) {
    s = EFI_OUT_OF_RESOURCES;
    goto exit;
  }

  memcpy (&S->FormSetP->Id, &FormSetOp->Guid, sizeof (EFI_GUID));
  S->FormSetP->Title = FormSetOp->FormSetTitle;
  S->FormSetP->Help = FormSetOp->Help;
  S->FormSetP->ClassIdCount = ClassGuidCount;
  memcpy (S->FormSetP->ClassIds, ClassGuid, ClassGuidCount * sizeof (EFI_GUID));

  if (!SysListAddTail (&S->FormPkgP->FormSets, S->FormSetP)) {
    SysDelete (S->FormSetP);
    S->FormSetP = NULL;
    s = EFI_OUT_OF_RESOURCES;
    goto exit;
  }

  s = EFI_SUCCESS;
exit:
  SysEmpty (&Guid);
  return s;
}

This function checks to make sure that we aren't already inside of a form set scope. Also, since this opcode can be variable sized (with 0-3 class GUIDs), we also need to make sure that the opcode has a valid size. Then the function creates a new Form Set container and then populates the members with data from the IFR FORM_SET opcode. The Form Package parsing state is updated with the pointer to the Form Set container. Finally, the Form Set container is added to the list maintained as a part of the Form Package container.

UEFI_HII_FORM_SET_P

The UEFI_HII_FORM_SET_P object acts as the Form Set container.

typedef struct _UEFI_HII_FORM_SET_P {
  SYS_OBJ Obj;

  EFI_GUID Id;
  int ClassIdCount;                     // number of entries in ClassIds
  EFI_GUID ClassIds[3];
  EFI_STRING_ID Title;
  EFI_STRING_ID Help;
  EFI_IMAGE_ID Image;
  EFI_ANIMATION_ID Animation;

  SYS_LIST_O Forms;                     // list of Form containers.
  SYS_LIST_O VarStores;                 // list of Variable Store containers.
  SYS_LIST_O DefaultStores;             // list of Default Store containers.
  SYS_MAP_U16P Questions;               // question id <-> Statement container
} UEFI_HII_FORM_SET_P;

typedef UEFI_HII_FORM_SET_P UefiHiiFormSetP;

Most of these members map directly to the IFR FORM_SET opcode, like the GUID (Id), the class GUIDs (ClassIds and ClassIdCount), the title text (Title) and the help text (Help). In addition, as we process the opcodes inside the Form Set scope, we will update the attributes of the Form Set container.  For example, the IFR IMAGE opcode will update the Image member.

There are four types of objects where the identifier is only unique within a form set: forms, variable stores, default stores and questions. Thus, there are three object lists (forms, variable stores, default stores) and one map (for questions). The questions are handled separately because, although the identifier is unique within the form set, they are also a part of the forms.

UefiHiiParseFormFormSetEndOp()

Now, when we reach the end of the Form Set scope, the function UefiHiiParseFormFormSetEndOp() is called.

EFI_STATUS
UefiHiiParseFormFormSetEndOp (
  IN CONST EFI_IFR_OP_HEADER *Op,
  IN OUT UEFI_HII_FORM_PKG_STATE *S
  )
{
  S->FormSetP = NULL;
  return EFI_SUCCESS;
}

Pretty simple, huh? Just update the Form Package parsing state to NULL, indicating that there is no active form set.

Variable Stores

Variable stores describe a virtual buffer used for configuration setting storage for one or more questions. There are three types of variable stores: buffer, EFI variable and name/value. Each of them has an identifier, a name and a GUID.

EFI_IFR_VARSTORE, EFI_IFR_VARSTORE_EFI and EFI_IFR_VARSTORE_NAME_VALUE

There are actually three separate opcodes, one for each type of variable store.

typedef struct {
  EFI_IFR_OP_HEADER Header;
 
  EFI_GUID Guid;
  EFI_VARSTORE_ID VarStoreId;
  UINT16 Size;
//UINT8 Name[];
} EFI_IFR_VARSTORE;

For the buffer variable store, there is a variable store identifier (VarStoreId) associated with a GUID (Guid) and name (Name). In addition, buffer variable stores have to specify the size, in bytes (Size).

typedef struct _EFI_IFR_VARSTORE_NAME_VALUE {
  EFI_IFR_OP_HEADER Header;
  EFI_VARSTORE_ID VarStoreId;
  EFI_GUID Guid;
} EFI_IFR_VARSTORE_NAME_VALUE;

For the name/value variable store, there is a variable store identifier (VarStoreId) associated with a GUID (Guid). There is also a name, but that is provided by the question header rather than being embedded in the variable store opcode.

typedef struct _EFI_IFR_VARSTORE_EFI {
  EFI_IFR_OP_HEADER Header;

  EFI_VARSTORE_ID VarStoreId;
  EFI_GUID Guid;
  UINT32 Attributes
  UINT16 Size;
//UINT8 Name[];
} EFI_IFR_VARSTORE_EFI;

For the EFI variable store, there is a variable store identifier (VarStoreId) associated with a GUID (Guid) and name (Name). In addition, EFI variable stores have to specify the size, in bytes (Size) and the variable attributes (Attributes).

UefiHiiParseFormVarStoreOp()

The Variable Store container is used for all types of variable stores. For buffer variable stores, here is the function:

EFI_STATUS
UefiHiiParseFormVarStoreOp (
  IN EFI_IFR_OP_HEADER *Op,
  IN OUT UEFI_HII_FORM_PKG_STATE *S
  )
{
  EFI_STATUS s;
  EFI_IFR_VARSTORE *VarStoreOp;
  UEFI_HII_VAR_STORE_P *VarStoreP;
  SYS_LIST_POS pos;
  SYS_STRA Guid;
  UINT32 i;

  SysStrAInit (&Guid);
  VarStoreOp = (EFI_IFR_VARSTORE *) Op;

  if (!UefiHiiOpIsValid (Op, S) ||
      !UefiHiiOpInFormSet (Op, S) ||
      !UefiHiiOpNotInForm (Op, S)) {
    return EFI_INVALID_PARAMETER;
  }

  for (i = sizeof (EFI_IFR_VARSTORE); i < Op->Length; i++) {
    if (((UINT8 *)Op)[i] == 0x00) {
      break;
    }
  }
  if (i == Op->Length) {
    return EFI_INVALID_PARAMETER;
  }
  if (VarStoreOp->Size == 0) {
    return EFI_INVALID_PARAMETER;
  }
  if (VarStoreOp->VarStoreId == 0) {
    return EFI_INVALID_PARAMETER;
  }

  ...dump out debug information...

  for (pos = NULL; SysListGetNext (&S->FormSetP->VarStores, &pos, &VarStoreP);) {
    if (!SysIsValid (VarStoreP)) {
      continue;
    }
    if (VarStoreP->Id == VarStoreOp->VarStoreId) {
      s = EFI_SUCCESS;
      goto exit;
    }
  }

  VarStoreP = SysNew (UefiHiiVarStoreP);
  if (VarStoreP == NULL) {
    s = EFI_OUT_OF_RESOURCES;
    goto exit;
  }

  VarStoreP->Type = Op->OpCode;
  VarStoreP->Id = VarStoreOp->VarStoreId;
  memcpy (&VarStoreP->Guid, &VarStoreOp->Guid, sizeof (EFI_GUID));
  VarStoreP->Size = VarStoreOp->Size;
  SysStrACopyAStr (&VarStoreP->Name, (char *)(VarStoreOp + 1));

  if (!SysListAddTail (&S->FormSetP->VarStores, VarStoreP)) {
    s = EFI_OUT_OF_RESOURCES;
    goto exit;
  }

  s = EFI_SUCCESS;
exit:
  SysStrAEmpty (&Guid);
  return s;
}

So this function checks to make sure that opcode is formatted correctly, including the size of the opcode. It also checks that there is only one variable store with the given variable store identifier in the form set. Then it creates the new Variable Store container from the IFR VARSTORE opcode and adds it to the current Form Set container.

UefiHiiParseFormVarStoreEfiOp()

Now let's look at the function for EFI variable stores.

EFI_STATUS
UefiHiiParseFormVarStoreEfiOp (
  IN EFI_IFR_OP_HEADER *Op,
  IN OUT UEFI_HII_FORM_PKG_STATE *S
  )
{
  EFI_STATUS s;
  EFI_IFR_VARSTORE_EFI *VarStoreOp;
  UEFI_HII_VAR_STORE_P *VarStoreP;
  SYS_LIST_POS pos;
  SYS_STRA Guid;
  UINT32 i;

  SysStrAInit (&Guid);
  VarStoreOp = (EFI_IFR_VARSTORE_EFI *) Op;

  if (!UefiHiiOpIsValid (Op, S) ||
      !UefiHiiOpInFormSet (Op, S)) {
    return EFI_INVALID_PARAMETER;
  }

  for (i = sizeof (EFI_IFR_VARSTORE_EFI); i < Op->Length; i++) {
    if (((UINT8 *)Op)[i] == 0x00) {
      break;
    }
  }
  if (i == Op->Length) {
    return EFI_INVALID_PARAMETER;
  }

  if (VarStoreOp->Size == 0) {
    return EFI_INVALID_PARAMETER;
  }
  if (VarStoreOp->VarStoreId == 0) {
    return EFI_INVALID_PARAMETER;
  }
  if (VarStoreOp->Attributes == 0) {
    return EFI_INVALID_PARAMETER;
  }

  ...dump debug information...

  for (pos = NULL; SysListGetNext (&S->FormSetP->VarStores, &pos, &VarStoreP);) {
    if (!SysIsValid (VarStoreP)) {
      continue;
    }
    if (VarStoreP->Id == VarStoreOp->VarStoreId) {
      s = EFI_SUCCESS;
      goto exit;
    }
  }

  VarStoreP = SysNew (UefiHiiVarStoreP);
  if (VarStoreP == NULL) {
    s = EFI_OUT_OF_RESOURCES;
    goto exit;
  }

  VarStoreP->Type = Op->OpCode;
  VarStoreP->Id = VarStoreOp->VarStoreId;
  memcpy (&VarStoreP->Guid, &VarStoreOp->Guid, sizeof (EFI_GUID));
  VarStoreP->Size = VarStoreOp->Size;
  VarStoreP->Attribs = VarStoreOp->Attributes;
  SysStrACopyAStr (&VarStoreP->Name, (char *)(VarStoreOp + 1));

  if (!SysListAddTail (&S->FormSetP->VarStores, VarStoreP)) {
    s = EFI_OUT_OF_RESOURCES;
    goto exit;
  }

  s = EFI_SUCCESS;
exit:
  SysStrAEmpty (&Guid);
  return s;
}

This is very similar to the buffer variable store, except that the type is set with the IFR VARSTORE_EFI opcode value.

UefiHiiParseFormVarStoreNameValueOp()

Now we try the same trick for name/value variable stores. Here is the function:

EFI_STATUS
UefiHiiParseFormVarStoreNameValueOp (
  IN EFI_IFR_OP_HEADER *Op,
  IN OUT UEFI_HII_FORM_PKG_STATE *S
  )
{
  EFI_STATUS s;
  EFI_IFR_VARSTORE_NAME_VALUE *VarStoreOp;
  UEFI_HII_VAR_STORE_P *VarStoreP;
  SYS_LIST_POS pos;
  SYS_STRA Guid;

  SysStrAInit (&Guid);
  VarStoreOp = (EFI_IFR_VARSTORE_NAME_VALUE *) Op;

  if (!UefiHiiOpIsValid (Op, S) ||
      !UefiHiiOpInFormSet (Op, S)) {
    return EFI_INVALID_PARAMETER;
  }

  if (VarStoreOp->VarStoreId == 0) {
    return EFI_INVALID_PARAMETER;
  }

  ...dump debug information...

  for (pos = NULL; SysListGetNext (&S->FormSetP->VarStores, &pos, &VarStoreP);) {
    if (!SysIsValid (VarStoreP)) {
      continue;
    }
    if (VarStoreP->Id == VarStoreOp->VarStoreId) {
      s = EFI_SUCCESS;
      goto exit;
    }
  }

  VarStoreP = SysNew (UefiHiiVarStoreP);
  if (VarStoreP == NULL) {
    s = EFI_OUT_OF_RESOURCES;
    goto exit;
  }

  VarStoreP->Type = Op->OpCode;
  VarStoreP->Id = VarStoreOp->VarStoreId;
  memcpy (&VarStoreP->Guid, &VarStoreOp->Guid, sizeof (EFI_GUID));

  if (!SysListAddTail (&S->FormSetP->VarStores, VarStoreP)) {
    s = EFI_OUT_OF_RESOURCES;
    goto exit;
  }

  s = EFI_SUCCESS;
exit:
  SysStrAEmpty (&Guid);
  return s;
}

Like the two previous functions, the opcode is checked for validity. Then a new Variable Store container is created from the IFR VARSTORE_NAME_VALUE opcode and added to the Form Set.

UEFI_HII_VAR_STORE_P

The Variable Store container structure (UEFI_HII_VAR_STORE_P) describes a variable store.

typedef struct _UEFI_HII_VAR_STORE_P {
  SYS_OBJ Obj;

  UINT8 Type;                     // VARSTORE, VARSTORE_EFI, VARSTORE_NAME_VALUE
  EFI_VARSTORE_ID Id;    
  EFI_GUID Guid;                  // variable store GUID
  SYS_STRA Name;
  UINT16 Size;                    // for VARSTORE
  UINT32 Attribs;                 // for VARSTORE_EFI
} UEFI_HII_VAR_STORE_P;

typedef UEFI_HII_VAR_STORE_P UefiHiiVarStoreP;

The Type describes the type of Variable Store: buffer (VARSTORE), EFI variable (VARSTORE_EFI) and name/value (VARSTORE_NAME_VALUE). The variable stores all have at least a variable store identifier (Id) and a GUID (Guid). The name (Name) and size (Size) is used for buffer and EFI variable stores. The attributes (Attribs) are only used for EFI variable stores.  

Conclusion

So far, we've looked at how to parse the form sets and the associated variable stores. Next time we'll look at forms and default stores. As you can see, each major IFR object type has its own container.

HOW TO: Disassembling the UEFI HII Database (Part 3)

Now we'll start to get into the real details. The next section of code disassembles the Forms package. The Forms package is encoded as a series of variable-length data structures called opcodes. This encoding is called Internal Forms Representation or IFR.

Opcodes

Each opcode has the following structure:

First, there is the Opcode, which is an enumerated value that specifies what type of object is being described. Then there is a single Scope bit, which describes whether there are any nested objects. Then there is the Length of the whole opcode (in bytes), including the opcode header and any Optional Opcode Data. Some opcodes do not have any optional data. Others always have it. In all cases, the next opcode is always Length bytes from the current opcode, and the last byte of the last opcode will always align with the end of the Forms package.

Pretty simple. Even if you don't know what an opcode does or what it means, you can just skip it. But there are two wrinkles: GUIDed opcodes and scoped opcodes.

GUIDed Opcodes

Opcode 0x5f does not have any specified meaning, but it does have a specified structure.

This was designed as a get-out-of-jail-free card for forms browsers, so that they could implement extended functionality without risking compatibility problems. Each vendor that would like to provided extended functionality simply creates their own GUID and then modify their browser to do something different. Other browsers will not recognize the GUID and simply skip the opcode.

In fact, this capability is already used in the EDK2 implementation of UEFI found on tianocore.org. If you are interested, look at MdeModulePkg/Include/Guid/MdeModuleHii.h.

Scoped Opcodes

Then there are scoped opcodes. The Scope bit in the opcode header says that all opcodes which follow are nested within--or, in the scope of--this opcode until a matching END opcode (0x29) is found. Theoretically, there is no limit to the number of levels of nesting, but practically it is limited to 10 or so.

When one opcode is nested (or in the scope) of another opcode, it is called a child opcode and the other is called its parent opcode. Child opcodes somehow modify or augment the parent opcode. So, question-style opcodes (numeric, string, etc.) are found in the scope of a form opcode.

When parsing, it is important to keep track of the different scopes, because some opcodes can be used in different contexts. For example, an image opcode (which provides a bitmap) can be found as a child of a form set opcode, a form opcode, various statement opcodes and the one-of-option opcode.

UefiHiiParseFormPkg()

Form Packages are parsed by the function UefiHiiParseFormPkg(), which creates a Form Package object and then goes into a big loop that processes all all of the opcode structures, one by one.When processing the opcodes, one of three things happens:

1. A new container is created for the object that opcode represents
2. An existing container is updated with new information or
3. The Form Package parsing state is updated.

Here is the function with the less interesting bits hidden for clarity:

 

EFI_STATUS

UefiHiiParseFormPkg (

  IN CONST EFI_IFR_OP_HEADER *Op,

  IN UINT32 Size,

  IN UEFI_HII_PACKAGE_P *PkgP,

  OUT SYS_OBJ **PkgData

  )

{

  UINT8 *OpData;

  EFI_STATUS s;

  UEFI_HII_FORM_PACKAGE_P *FormPkgP;

  UEFI_HII_FORM_PKG_STATE S;

  EFI_IFR_OP_HEADER *TempOp;

 

 

 

  ...do error checking on the input...

 

  OpData = (UINT8 *)Op;

  *PkgData = NULL;

 

  //

  // Create the Form Package container.

  //

  FormPkgP = SysNew (UefiHiiFormPackageP);

  if (FormPkgP == NULL) {

    return EFI_OUT_OF_RESOURCES;

  }

  *PkgData = &FormPkgP->Obj;

 

 

  FormPkgP->Pkg = PkgP;                 // track the parent package.

 

  ...initialize the Form Package parsing state to defaults...

 

 

 

  while (Size >= sizeof (EFI_IFR_OP_HEADER) && Size >= Op->Length) {

    S.OpOffset = (UINT32)((UINT8 *)Op - OpData);

 

    ...dump out debug information...

 

    s = EFI_UNSUPPORTED;

    if (UefiFormPkgParse[Op->OpCode].OpCodeName != NULL) {

      if (UefiFormPkgParse[Op->OpCode].StartParse != NULL) {

        if (Op->Length < UefiFormPkgParse[Op->OpCode].OpCodeMinSize) {

          SYSINFO (

 

 

            "S2006 : 0x%08x : '%s' : Opcode size error. "

            "Expected at least %d bytes. Found %d bytes.\n",

            S.OpOffset,

            UefiHiiDumpFormOp (Op),

            UefiFormPkgParse[Op->OpCode].OpCodeMinSize,

            Op->Length);

        } else {

          UefiFormPkgParse[Op->OpCode].StartParse (Op, &S);

        }

      }

    } else {

      SysTrace ("Unknown IFR Opcode: 0x%02x\n", Op->OpCode);

    }

 

    //

    // If the opcode has scope, then push the current parent opcode pointer on

    // the stack. If the opcode is an IFR END opcode, then process the end of

    // scope and pop the current parent opcode pointer from the stack.

    //

    // There are some opcodes that *could* have scope, but did not this time.

    // We treat this as if we had found an END opcode, so that clean up is

    // consistent.

    //

    if (Op->OpCode == EFI_IFR_END_OP) {

      if (Op->Scope) {

        SYSINFO ("END: Opcode cannot have Scope bit set. Ignored\n");

      }

 

      if (SysListIsEmpty (&S.Scopes)) {

        SYSINFO ("END: Unexpected without matching Scoped opcode.\n");

        s = EFI_INVALID_PARAMETER;

        goto exit;

      }

      TempOp = S.ParentOp;

      S.ParentOp = (EFI_IFR_OP_HEADER *)SysListRemoveTail (&S.Scopes);

      if (UefiFormPkgParse[TempOp->OpCode].OpCodeName != NULL &&

          UefiFormPkgParse[TempOp->OpCode].EndParse != NULL) {

        s = UefiFormPkgParse[TempOp->OpCode].EndParse (TempOp, &S);

        if (EFI_ERROR (s)) {

          gUefiHiiTraceIndent--;

          goto exit;

        }

      } 

      gUefiHiiTraceIndent--;

    } else if (Op->Scope) {

      SysListAddTail (&S.Scopes, S.ParentOp);

      S.ParentOp = (EFI_IFR_OP_HEADER *)Op;

      gUefiHiiTraceIndent++;

    } else {

      if (UefiFormPkgParse[Op->OpCode].OpCodeName != NULL && 

          UefiFormPkgParse[Op->OpCode].EndParse != NULL) {

        s = UefiFormPkgParse[Op->OpCode].EndParse (Op, &S);

        if (EFI_ERROR (s)) {

          goto exit;

        }

      }

    }

     

    //

    // Move to the next opcode.

    //

    Size -= Op->Length;

    Op = (EFI_IFR_OP_HEADER *)((UINT8 *)Op + Op->Length);

  }

 

 

  s = EFI_SUCCESS;

exit:

 

 

  ...clean up parsing data structures...

  return EFI_SUCCESS;

}

There are three key data structures used in this function:

1. S. S is a structure of type UEFI_HII_FORM_PKG_STATE, which contains the current parsing state. It keeps track of the current scope and important current objects, like the current question or the current form. It gets passed around to the parsing functions and updated as the opcodes are processed. We will talk about this a bit more, below.
2. UefiFormPkgParse[]. This array contains one entry for every possible opcode value, from 0x00 to 0xff. It contains a pointer to the name of the opcode, a pointer the function to call when an opcode is first processed and a function to call when an opcode's scope is closed. For the purposes of this function, an opcode's scope is closed when (a) it has the Scope bit set and then a matching END opcode is found or (b) it does not have the Scope bit set. We do it this way because many opcodes can have child opcodes or not, but some processing has to wait until all child opcodes (if any) have been processed.
3. FormPkgP. Pointer to the Form Package container object. This is the pointer that is placed into the PkgDataP member of the current Package container for Forms Packages.   

UEFI_HII_FORM_PKG_STATE

This structure holds the current parsing state:

typedef struct _UEFI_HII_FORM_PKG_STATE {
  UINT32 OpOffset;                     
  EFI_IFR_OP_HEADER *ParentOp;         
  UEFI_HII_FORM_PACKAGE_P *FormPkgP;   

  SYS_LIST_O DisableIfP;             
  SYS_LIST_O SuppressIfP;            
  SYS_LIST_O GrayOutIfP;               

  UEFI_HII_FORM_SET_P *FormSetP;     
  UEFI_HII_FORM_P *FormP;            
  UEFI_HII_STMT_P *StmtP;            
  UEFI_HII_OPTION_P *OptionP;        
  UEFI_HII_DEFAULT_P *DefaultP;        

  SYS_LIST Scopes;                   
  SYS_LIST_O ExprP;                  
  SYS_LIST_O ValueP;                   
} UEFI_HII_FORM_PKG_STATE;

The OpOffset field records the offset of the current opcode from the beginning of the forms package. This is used to help display errors or debug information. The ParentOp points to the parent opcode whose scope contains the current opcode, or NULL if it is at the top level. The FormPkgP points to the Form Package container associated with the forms package being parsed.

The next three object lists are used to hold expressions for disabling, suppressing or graying-out various other IFR objects. In IFR, these are actually parent objects and objects like questions and statements and one-of-options are inside their scope. But I prefer to think of these expressions as attributes of the question, so what I do is keep a running list of all of the active parent expressions. Then, when I find a question or a statement or some other IFR object, I make a copy of the active expressions in my container. This is not only my preference (to make these attributes), but it also simplifies some sorts of parent-child error checking.

The next five members point to the containers for important objects that are currently active. For example, once we process the form set opcode and create the Form Set container, we place the pointer in FormSetP. When we leave the form set opcode's scope, FormSetP is set to NULL. It so happens that forms must be in form sets, statements must be in forms and defaults/options must be in certain statements.

The Scopes list is a first-in-last-out stack that contains the pointers to parent IFR opcodes. Each time we enter a scope, the current parent opcode (ParentOp) is pushed on to the stack and each time we exit a scope, the top element in the stack is popped into ParentOp.

The ExprP object list contains the current expression stack. Expressions in IFR operate on an expression stack and are encoded in prefix-notation (operands are encountered before operators). I prefer to store these in a tree structure instead (that is, an operator with zero or more operands). So, when we are parsing expressions, we pop zero or more expression containers from the expression stack and push the newly created expression container.

The ValueP object list contains the results of the most recent IFR value opcode, which will then be attached either to a one-of container or a statement container.

UEFI_HII_FORM_PACKAGE_P

This structure holds information about the Form Package:

typedef struct _UEFI_HII_FORM_PACKAGE_P {
   SYS_OBJ Obj;            

   UEFI_HII_PACKAGE_P *Pkg;
   SYS_LIST_O FormSets;    
} UEFI_HII_FORM_PACKAGE_P;

typedef UEFI_HII_FORM_PACKAGE_P UefiHiiFormPackageP;

The top-most encoded IFR objects in form package are always form sets. So FormSets is an object list containing form set containers for all IFR form set objects, in the order which they were encountered in the Form Package.

Conclusion

Now that we've made it to actually parsing opcodes, things are pretty straight forward. Now there will be either a container object for whatever is encountered in the IFR, or the IFR contents will modify previously existing container objects. There are containers for form sets, forms, statements, one-of options, values, expressions, variable stores, default stores and defaults. In the next article, we will delve into form sets and forms.

HOW TO: Disassembling the UEFI HII Database (Part 2)

In this series of articles, we are examining the UEFI HII database, a repository for all sorts of user-interface resources used by UEFI applications, including forms, strings, fonts, images and animations. UEFI firmware also provides a built-in API for displaying the forms and interactin the the user called the Form Browser2 protocol.

In part 1, we looked at the top-most layer of our disassembler, which retrieved all of the installed resources from the database. The resources are binary-encoded into variable-length data structures called Packages and then grouped together into Package Lists. Part 1 handled the Package Lists and now we will take a look at the Package structure and parsing code.

UEFI HII Packages

The encoding for UEFI HII Packages is deceptively simple: there is a header and then a body. The header describes what type of package it is and its size, in bytes.

typedef struct {
  UINT32  Length:24;
  UINT32  Type:8;
// UINT8  Data[...];
} EFI_HII_PACKAGE_HEADER;


So, likewise, the basic data structure for holding UEFI HII Packages is quite simple:

#define UEFI_HII_PACKAGE_SIGNATURE 0x5F504855 // "UHP_"
typedef struct _UEFI_HII_PACKAGE_P {
  SYS_OBJ Obj;                         

  UINT8 Type;                           // See EFI_HII_PACKAGE_x
  UEFI_HII_PACKAGE_LIST_P *PkgList;     // package list parent.
  SYS_OBJ *PkgData;                     // ptr to data specific to package type.
} UEFI_HII_PACKAGE_P;

typedef UEFI_HII_PACKAGE_P UefiHiiPackageP;
The Obj member provides the basic object-oriented functionality of the SysLib, including function pointers to initialization, emptying, copying and validity-checking functions.

The Type member matches the Type member of the IFR Package header. There are currently 8 types of packages as well as a GUIDed type for expansion and 32 reserved types for system vendors.

The PkgList member points to the structure that represents the package list which contains this package. If the package was parsed independently, as would happen when disassembling a file  containing only package binary-encoded data, this would be NULL.

The PkgData member points to an object that represents the contents of the package. The exact object depends on the Type member. So for forms, this is a pointer to a UEFI_HII_FORM_PACKAGE_P. By using the object pointer, we can handle copying and memory management without knowing the exact structure.
 

UefiHiiParsePkg()

In part 1, we saw that UefiHiiParsePkgList() walks through all packages in a package list and calls this function to handle actually parsing the package data and converting it into our structure. Essentially this is a factory function which creates a package object. It then uses the Type value to call another parsing function which processes the package body and records the pointer to the resulting data object.

EFI_STATUS
UefiHiiParsePkg (
  IN EFI_HII_PACKAGE_HEADER *Pkg,
  OUT UEFI_HII_PACKAGE *PHandle
  )
{
  EFI_STATUS s;
  UINT8 *PkgData;
  UINT32 PkgDataSize;
  UEFI_HII_PACKAGE_P *PkgP;
 
  if (Pkg == NULL || PHandle == NULL) {
    return EFI_INVALID_PARAMETER;
  }
  if (Pkg->Length < sizeof (EFI_HII_PACKAGE_HEADER)) {
    return EFI_INVALID_PARAMETER;
  }
 
  ...
  PkgP = SysNew (UefiHiiPackageP);
  if (PkgP == NULL) {
    return EFI_OUT_OF_RESOURCES;
  }
  *PHandle = (UEFI_HII_PACKAGE)PkgP;
 
  UefiHiiSetPkgType (*PHandle, (UINT8) Pkg->Type);
 
  PkgData = (UINT8 *)(Pkg + 1);
  PkgDataSize = Pkg->Length - sizeof (EFI_HII_PACKAGE_HEADER);
  switch (Pkg->Type) {
 
  case EFI_HII_PACKAGE_FORMS:
    s = UefiHiiParseFormPkg (
          (EFI_IFR_OP_HEADER *)PkgData,
          PkgDataSize,
          PkgP,
          &PkgP->PkgData
          );
    break;
 
  case EFI_HII_PACKAGE_STRINGS:
    s = UefiHiiParseStringPkg (
          (EFI_HII_STRING_PACKAGE_HDR *)Pkg,
          PkgDataSize,
          PkgP,
          &PkgP->PkgData
          );
    break;
 
  case EFI_HII_PACKAGE_SIMPLE_FONTS:
    s = UefiHiiParseSimpleFontPkg (
          (EFI_HII_SIMPLE_FONT_PACKAGE_HDR *)Pkg,
          PkgDataSize,
          PkgP,
          &PkgP->PkgData
          );
    break;

  case EFI_HII_PACKAGE_ANIMATIONS:
    s = UefiHiiParseAnimationPkg (
          (EFI_HII_ANIMATION_PACKAGE_HDR *)Pkg,
          PkgDataSize,
          PkgP,
          &PkgP->PkgData
          );
    break;

  case EFI_HII_PACKAGE_IMAGES:
    s = UefiHiiParseImagePkg (
          (EFI_HII_IMAGE_PACKAGE_HDR *)Pkg,
          PkgDataSize,
          PkgP,
          &PkgP->PkgData
          );
    break;
  case EFI_HII_PACKAGE_FONTS: 
    s = UefiHiiParseFontPkg (
         (EFI_HII_FONT_PACKAGE_HDR *)Pkg,
         PkgDataSize,
         PkgP,
         &PkgP->PkgData
         );
    break;
  case EFI_HII_PACKAGE_DEVICE_PATH:
    s = EFI_SUCCESS;
    break;
 
  case EFI_HII_PACKAGE_END:
    s = EFI_SUCCESS;
    break;
 
  default:
    s = EFI_INVALID_PARAMETER;
    break;
  }
 
  if (EFI_ERROR(s)) {
    SysDelete (PkgP);
  }
  return s;
}

Conclusion

I have described the actual format of string, font and image packages before, but never the forms. If you look carefully above, you can see that the FORMS package parsing function is slightly different than the others. The string, font, image and animation packages all start with fixed headers which are essentially supersets of the package header. The forms package does not. It uses the standard package header but then contains a series of variable length "opcodes". And what is IFR anyway? More on this next time, where we will go into depth into how forms are encoded.

HOW TO: Disassembling the UEFI HII Database (Part 1)

This article is the first in a series that talks about UEFI's Human Interface Infrastructure (HII) database. The HII Database is the repository for all sorts of user-interface related information in a platform, including forms, strings, bitmaps, fonts and keyboard layouts. Within UEFI, these resources are used primarily to present configuration information to a user. One example is the setup application, common to most PC BIOS firmware implementations. But that is not all. UEFI also uses these resources to implement the Driver Health and User Identification infrastructure. And, of course, our applications can use these.

I'm going to show you the HII database by using the source code to a tool (hiidd) that parses the HII database contents into structures and then displays the information. This tool is not merely a disassembler, but also a foundation for further UEFI tools. It uses some of the SysLib that I described previously.

main()

So let's jump in to main:

int

EFIAPI

main (

  IN int Argc,

  IN char **Argv

  )

{

  int ret;

 

  ret = 0;

  InitCmdLine();

  ret = ParseCmdLine (Argc, Argv);

  if (ret != 0) {

    goto error;

  }

  if (gPackageListFromHiiDatabase) {

    verbosePhase("Read package lists from HII database.\n");

    ret = ReadHiiDatabase();

  } else if (gPackageListFromFiles) {

    verbosePhase("Read package lists from files.\n");

    ret = ReadPackageListsFromFiles();

  } else if (gPackageFromFiles) {

    verbosePhase("Read pacakges from files.\n");

    ret = ReadPackagesFromFiles();

  }

  DumpPackageLists();
error:

  ShutCmdLine();

  return ret;

}

 

Pretty standard. We parse the command-line, read the HII database either from UEFI or from a file and then dump it out. The tool supports the following command-line options: 

• -hiidb - Read the package lists from the HII Database on the machine.
• -packagelist file-name - Read the package list from the file file-name. The file has the package list format as described in the UEFI specification.
• -package file-name - Read in an individual package from the file file-name. The file has the package fromat as described in the UEFI specification.
• -verbose, -v1, -v2, -v3 - Turn on the level of informational output provided by the tool. 1 = phases, 2 = actions, 3 = the kitchen sink. There are three functions: verbosePhase(), verboseAction() and verboseInfo() which will only display the string if the verbosity level is set to the corresponding level.

ReadHiiDatabase()

 Now let's take a look at the main HII Database parsing code:

int
ReadHiiDatabase (void)
{
  EFI_STATUS s;
  EFI_HII_PACKAGE_LIST_HEADER *PackageLists;
  EFI_HII_PACKAGE_LIST_HEADER *PackageList;
  UINTN PackageListSize;
  int PackageListIndex;
  SYS_STRA guid;
  int ret;
  UEFI_HII_PACKAGE_LIST PLHandle;

  SysStrAInit (&guid);

  //
  // All of the package lists are exported into one big buffer.
  //
  s = UefiHiiExportPackageListsA (
        NULL,
        &PackageLists,
        &PackageListSize
        );
  if (EFI_ERROR (s)) {
    ret = 1;
    goto exit;
  }

  PackageListIndex = 0;
  PackageList = PackageLists;
  while (PackageListSize > sizeof (EFI_HII_PACKAGE_LIST_HEADER)) {
    verboseInfo ("Package List #%d\n",
                 PackageListIndex
                 );
    verboseInfo ("  Offset: 0x%08x\n",
                 (UINT32)((UINT8 *)PackageList - (UINT8 *)PackageLists));

    SysStrAFromGuid (&guid, &PackageList->PackageListGuid);
    verboseInfo ("  GUID:   %s\n", SysStrAGetData(&guid));
    if (PackageList->PackageLength > PackageListSize) {
      printf ("error: package list extends beyond end of buffer. "
              "%d bytes in package list. %d bytes in buffer.\n",
              PackageList->PackageLength,
              PackageListSize
              );
      ret = 1;
      goto exit;
    }
    s = UefiHiiParsePkgList (
          PackageList,
          &PLHandle
          );
    if (EFI_ERROR(s)) {
      printf ("error: unable to parse package list.\n");
      ret = 1;
      goto exit;
    }
    if (!SysArrayAppend (&gPackageLists, &PLHandle)) {
      printf ("fatal: out of memory.\n");
      exit (1);
    }

    PackageListIndex++;
    PackageListSize -= PackageList->PackageLength;
    PackageList =
      (EFI_HII_PACKAGE_LIST_HEADER *)
      ((UINT8*)PackageList + PackageList->PackageLength);
  }
  if (PackageListSize != 0) {
    printf("error: HII database array of package lists did not "
           "end on an even boundary.\n");
    ret = 1;
    goto exit;
  }
  ret = 0;
exit:
  SysStrAEmpty (&guid);
  return ret;
}
In this section, we use the library function UefiHiiExportPackageListsA to grab all of the HII package lists from the database into one big buffer. More details on that function later. Then the function UefiHiiParsePkgList() runs through the data, creates a package list handle for each package list found and adds that handle to an array. Each handle is associated with a single package list.

I use a handle here to abstract the relationship between the application and the actual data structures inside the parsing library. This allows me to refactor the code later without messing up the apps that depend on it.

Package Lists are really just big containers for zero or more Packages, identified by a GUID. The GUID is just an identifier that allows the user to uniquely identify a package list in the database. Packages, in turn, are containers for all sorts of interesting things, like Forms, Strings, Fonts, Images, Animations, Keyboard Layouts or OEM data. I've talked about some of these before in the early days of my blog.

UefiHiiParsePkgList()

Inside the UefiHiiParsePkgList(), each handle is actually a pointer to a private data structure constructed by the library:

typedef struct _UEFI_HII_PACKAGE_LIST_P {
  SYS_OBJ Obj;                     // standard object structure. must be first.

  EFI_GUID Id;                     // package list identifier.
  SYS_LIST_O Packages;             // list of packages in package list.
} UEFI_HII_PACKAGE_LIST_P;

#define UEFI_HII_PACKAGE_LIST_SIGNATURE 0x4C504855 // "UHPL"

typedef UEFI_HII_PACKAGE_LIST_P UefiHiiPackageListP;
This structure uses the same object model (SysObj) from my library, which means every instance contains function pointers to instances of Init(), Empty(), Copy(), IsValid() and Dump() as well as a signature so that we can validate object pointers. The member 'Packages' is an Object List container where each list entry is an Object. So when the List container is emptied, all the memory will be freed automatically.

So, each package list is parsed by this function, and a single object of type UEFI_HII_PACKAGE_LIST_P is created to represent it. The pointer is typecast into the package list handle type and returned. 

Here's the actual code:
EFI_STATUS
UefiHiiParsePkgList (
  IN EFI_HII_PACKAGE_LIST_HEADER *PkgList,
  OUT UEFI_HII_PACKAGE_LIST *PLHandle
  )
{
  EFI_STATUS s;
  EFI_HII_PACKAGE_HEADER *Pkg;
  UINT32 PkgListSize;
  UEFI_HII_PACKAGE PHandle;
  UINT32 PkgIndex;

  if (PkgList == NULL || PLHandle == NULL) {
    return EFI_INVALID_PARAMETER;
  }
  if (PkgList->PackageLength < sizeof (EFI_HII_PACKAGE_LIST_HEADER)) {
    return EFI_INVALID_PARAMETER;
  }

  ...

  *PLHandle = (UEFI_HII_PACKAGE_LIST) SysNew (UefiHiiPackageListP);
  if (*PLHandle == NULL) {
    return EFI_OUT_OF_RESOURCES;
  }
  memcpy (
   &((UEFI_HII_PACKAGE_LIST_P *)(*PLHandle))->Id,
   &PkgList->PackageListGuid,
   sizeof (EFI_GUID)
   );

  PkgIndex = 0;
  PkgListSize = PkgList->PackageLength - sizeof (EFI_HII_PACKAGE_LIST_HEADER);
  Pkg = (EFI_HII_PACKAGE_HEADER *)(PkgList + 1);
  while (PkgListSize >= sizeof (EFI_HII_PACKAGE_HEADER)) {
    ...   

    if (Pkg->Length > PkgListSize) {
      return EFI_VOLUME_CORRUPTED;
    }
    s = UefiHiiParsePkg (
          Pkg,
          &PHandle
          );
    if (EFI_ERROR(s)) {
      return s;
    }
   
    s = UefiHiiSetPkgPkgList (PHandle, *PLHandle);
    if (EFI_ERROR (s)) {
      return s;
    }

    PkgIndex++;
    PkgListSize -= Pkg->Length;
    Pkg = (EFI_HII_PACKAGE_HEADER *)((UINT8 *)Pkg + Pkg->Length);
  }
  if (PkgListSize != 0) {
    return EFI_VOLUME_CORRUPTED;
  }
  return EFI_SUCCESS;
}
After the package list header is the body of the package list, which consists of a series of variable-length package structures. Each fo the package structures contains is own length, so it is trivial to find the first one, parse it, mark it as belong to this package list (using UefiHiiSetPkgPkgList()) and then advancing the pointer to the next one. There is some error checking code to make sure we don't advance past the end of the buffer and that the end of the last package aligns exactly with the end of the package list body. The main work of parsing the individual packages is done using the function UefiHiiParsePkg().

Conclusion

At this point, we're just scratching the surface of the HII Database. Next week, we'll dive a little deeper into parsing the packages, introduce the UEFI_HII_PACKAGE_P object, and peek into what it means to parse the forms (IFR).

Extra Stuff: UefiHiiExportPackageListsA()

You don't need to read this part unless you are curious how my SysLib's HII database functions actually relate to the functions described in chapter 30 of the UEFI specification. This time, we talked about the function UefiHiiExportPackageListsA(), which acts as a handy wrapper for the HII Database function ExportPackageLists(). It takes care of the memory allocate details for you.

EFI_STATUS
UefiHiiExportPackageListsA (
  IN EFI_HII_HANDLE Handle,
  OUT EFI_HII_PACKAGE_LIST_HEADER **PackageList,
  OUT UINTN *PackageListSize
  )
{
  EFI_STATUS s;

  if (EFI_ERROR (_UefiHiiDatabaseProtocol())) {
    return EFI_NOT_FOUND;
  }

  if (PackageList == NULL || PackageListSize == NULL) {
    return EFI_INVALID_PARAMETER;
  }

  *PackageListSize = 0;
  s = gHiiDb->ExportPackageLists (
                gHiiDb,
                Handle,
                PackageListSize,
                *PackageList
                );
  if (!EFI_ERROR (s)) {
    return s;
  } else if (s != EFI_BUFFER_TOO_SMALL) {
    return s;
  }

  *PackageList = (EFI_HII_PACKAGE_LIST_HEADER *)malloc (*PackageListSize);
  if (*PackageList == NULL) {
    return EFI_BUFFER_TOO_SMALL;
  }
  s = gHiiDb->ExportPackageLists (
                gHiiDb,
                Handle,
                PackageListSize,
                *PackageList
                );
  return s;
}

The function ExportPackageLists() is from the HII Database protocol and takes all of the package lists from the HII Database and puts them in a big buffer. This is done with two calls. The first call returns how much size is actually needed. The second call actually gets the data.