UEFI HII (Part 11): Image Package Encoding

The UEFI specification describes a standard set of APIs for drawing bitmaps. The HII Image protocols (as well as the Graphics Output protocol) only deal with bitmaps as arrays of 32-bit pixels. But UEFI also describes a standard way that these bitmaps (or images, as the specification calls them) should be encoded as resources (or packages).

 

Within the packages, images are encoded in 5 different ways:

• 1-bit per pixel with palette. Each pixel in the image is encoded as a single bit within a byte. Each row of the image is padded to a byte boundary (as with font glyphs). Both the 0 and 1 values can be translated to a full 24-bit color value if a palette entry is provided.
• 4-bits per pixel with palette. Each pixel in the image is encoded as 4 bits within a byte. Each row of the image is padded to a byte boundary. Each of the 16 possible pixel values can be translated to a full 24-bit color value if a palette entry is provided.
• 8-bits per pixel with palette. Each of the 256 possible pixel values can be translated to a full 24-bit color value if a palette entry is provided.
• 24-bits per pixel. Each pixel takes up exactly 3 bytes, one each for red, green and blue.
• JPEG. Support is required for high (1:1:1) and medium (4:1:1) quality JPEG encoding for R/G/B. There are many other sub-types of JPEG encoding, such as gray-scale encoding for medical imaging, which are not required to be supported.

Fortunately, developers normally never need to deal directly with these formats because they are all translated by the HII Image protocol into the standard 32-bit-per-pixel format used throughout the UEFI specification, including the drawing functions of the  Graphics Output protocol.  Tools are used at build time to convert bitmap files in various formats into of the UEFI encodings.

 

Image packages are very similar in concept to the string packages described before. They always start with the header:

 

typedef struct _EFI_HII_IMAGE_PACKAGE_HDR {

  EFI_HII_PACKAGE_HEADER Header;

 

  UINT32 ImageInfoOffset;
  UINT32 PaletteInfoOffset;
} EFI_HII_IMAGE_PACKAGE_HDR;

 

There are two offsets: the first is to the series of image blocks which describe the images themselves. The second is to the palette information.

The palette information consists of zero or more palettes. Each palette is an array of 32-bit color values, assigned an index between 0 and 255. Images can refer to a palettte and can share a palette. Palettes are not required to carry all 2 (1-bit), 16 (4-bit) or 256 (8-bit) colors but rather only those actually necessary for the images carried in the package.

 

The image information consists of zero or more image blocks, terminated by a special image block of type IIBT_END. Images are encoded in ascending order by their image identifier, starting with the value 1. Each image block does one of the following:

• Associate a normal image with the current image identifier value.
• Assocate a transparent image with the current image identifier value. Transparent images specify that the color value 0 should not be drawn.
• Associate a previous image with the current image identifier value.
• Skip a specified number of image identifier values.

The following diagram shows a simple 1-bit image as it is encoded and then how it is translated using a palette.

Conclusion

Images are just one of the many types of HII-related resources supported by the UEFI specification. The images are included in package lists which can be attached as part of the PE/COFF resources or loaded as separate files or file sections. Next time we will start looking at the most interesting of the HII constructs: the form, which encapsulate configuration settings.

WHY: Why Do I Get Unresolved Externals For __allmul?

While I was working on my project this week, I kept running into the following error:

CLibApp.lib(String.obj) : error LNK2019: unresolved external symbol __allmul referenced in function _wcssize


CLibApp.lib(Cwd.obj) : error LNK2001: unresolved external symbol __allmul
 
This function appears nowhere in my code, nor does it appear in any of the EDK's code? So what's going on? It turns out that this function is one of several compiler support functions that are invoked explicitly by the Microsoft C/C++ compiler. In this case, this function is called whenever the 32-bit compiler needs to multiply two 64-bit integers together. The EDK does not link with Microsoft's libraries and does not provide this function.
 
So why don't all the other drivers and applications in the EDK generated unresolved externals, since they obviously do 64-bit math? The EDK authors skirted this problem by creating 64-bit math routines of their own, such as MultU64x64 and MultU64x32 and using these instead of the built-in C/C++ multiply (*) operator.
 
Are there other functions like this one? Sure, several more for 64-bit division, remainder and shifting.
 
Interestingly enough, the EDK does contain some Microsoft C/C++ compiler support. See CompilerStub.c where both memcpy and memset (from the C standard library) are defined. Why? It turns out that later versions of the compiler optimize certain code sequences by calling the library routines. Want a little stranger bit of trivia? The EfiCommonLib tries to optimize the SetMem function by special-casing a set to zero (i.e. SetMem (dest, count, 0)). But it turns out that the C compile convers both branches into a call to memset.
 
Anyway, here is my implementation of the multiplication routine. The Microsoft version is available, but has their license. The other versions available on the web look an awful lot like the Microsoft version, down to the comments, so I reverse engineered the calling convention (old-style STDCALL) and wrote it from scratch in MASM 9.0.
 
Tim
 
; allmul - 64-bit signed multiplication support function.



.586
.MODEL FLAT, C
.CODE

;
; FUNCTION NAME.
; _allmul
;
; FUNCTIONAL DESCRIPTION.
; This function is called by the Microsoft Visual C/C++ compiler for 32-
; bit executables to multiply two 64-bit integers and returning a 64-bit
; result. The X86 processors have only a 32-bit multiply instruction,
; thus the necessity for a library support function.
;
; The operands are divided into two 32-bit quantities. You can imagine
; that this works like simple 2-digit x 2-digit multiplication, except
; that each digit is 32-bits wide.
;
;   AB
; x CD
; ----
;   DB
;  DA0
;  CB0
; CA00
; ----
; RRRR
;
; You notice that the 3rd and 4th columns never will be used because the
; are the part of the result that is > 64-bits.
;
; R[0:31] = DB[0:31]
; R[32:63] = DB[32:63] + DA[0:31] + CB[0:31]
;
; There is a short cut, if both A and C are 0, then we can use the simple
; 32-bit instruction.
;
;
; ENTRY PARAMETERS.
;    multiplicand - Right-hand operator (CD)
;    multiplier - Left-hand operator (AB)
;
; EXIT PARAMETERS.
;    EDX:EAX - Result.
;


_allmul PROC NEAR USES ESI, multiplicand:QWORD, multiplier:QWORD

 MA EQU DWORD PTR multiplier [4]
 MB EQU DWORD PTR multiplier
 MC EQU DWORD PTR multiplicand [4]
 MD EQU DWORD PTR multiplicand

 mov eax, MA
 mov ecx, MC
 or  ecx, eax    ; both zero?
 mov ecx, MD
 .if zero?      ; yes, use shortcut.
   mov eax, MB
   mul ecx      ; EDX:EAX = DB[0:63].
 .else
   mov eax, MA
   mul ecx      ; EDX:EAX = DA[0:63].
   mov esi, eax ; ESI = DA[0:31].

   mov eax, MB
   mul MC       ; EDX:EAX = CB[0:63]
   add esi, eax ; ESI = DA[0:31] + CB[0:31]


   mov eax, MB
   mul ecx      ; EDX:EAX = BD[0:63]
   add edx, esi ; EDX = DA[0:31] + CB[0:31] + DB[31:63]
                ; EAX = DB[0:31]
 .endif


 ret 16 ; callee clears the stack.
_allmul ENDP


 END

HOW-TO: Debug The EDK's Windows-Hosted UEFI Environment

Last time we took a quick look at how to set up the Windows-hosted (NT32) UEFI environment provided by the EDK. The NT32 environment is very useful for debugging UEFI applications which aren't tied to specific hardware devices. So this week, I'll show how to add on debugging support.

This article assumes that you have already loaded the Visual Studio project to build the EDK's NT32 platform in C:\EDK as described previously.

1. Select the NT32 project in the Solution Explorer.

2. Select Project|Properties

3. In the "NT32 Properties Pages" select "Debugging"

4. Select "Command" and enter "SecMain.exe".
5. Select "Working Directory" and enter "c:\edk\sample\platform\nt32\uefi\ia32".

6. Select "Environment" and select the ...

7. An editor will pop up. In the editor box, enter the following and the click "OK". These are environment variables which govern how the emulator works, how much memory it uses, what virtual devices it has access to, etc. We will discuss these more in the next article.

EFI_WIN_NT_PHYSICAL_DISKS=a:RW;2880;512!g:RW;262144;512

EFI_WIN_NT_VIRTUAL_DISKS=FW;40960;512
EFI_WIN_NT_SERIAL_PORT=COM1!COM2
EFI_WIN_NT_GOP=Graphics Output Window 1!Graphics Output Window 2
EFI_WIN_NT_UGA=UGA Window 1!UGA Window 2
EFI_FIRMWARE_VOLUMES=..\Fv\FvRecovery.fd
EFI_WIN_NT_FILE_SYSTEM=.!%EDK_SOURCE%\Other\Maintained\Application\UefiShell\bin\ia32\Apps
EFI_MEMORY_SIZE=64!64
EFI_BOOT_MODE=1
EFI_WIN_NT_CPU_MODEL=Intel(R) Processor Model
EFI_WIN_NT_CPU_SPEED=3000

8. Click "OK"

Debugging

Now you are ready to actually debug. Press F5 (or select Debug|Start Debugging). It will always ask you if you want to rebuild the project, since we havne't added any source files which Visual Studio can use to determine if the project source has been changed. For now, choose Yes and then you will see the program begin to execute normally.

You can halt the execution of the emulated UEFI environment at any time by selecting Debug|Break All. Then you can set some breakpoints on a specific function using Debug|New Breakpoint|Break At Function and type in the function name. You can force a breakpoint by inserting a __debugbreak() into your code.

As the code executes you will notice that the Visual Studio window talks about various DLLs being loaded. The Windows-hosted environment actually loads your UEFI drivers and applications as DLLs (look in C:\Edk\Sample\Platform\Nt32\Uefi\Ia32). The actual main program is called SecMain.exe.

Conclusion
So now we're debugging. But what else can we do with the Windows-hosted (NT32) environment? Next time we'll look at how the environment can be configured.

HOW-TO: Set Up The EDK's Windows-Hosted UEFI Environment With Visual Studio 2008.

Since I'm working on a little research project of my own using UEFI applications, I thought I'd use the Windows-hosted UEFI (aka NT32) environment provided with the EDK. From previous experience, I know that the ability to debug applications using the Visual Studio environment speeds up my development time considerably. So I thought I'd share how I set up my environment. Then next time, I'll share how I set up for debugging.

This article assumes that you have Visual Studio 2008 installed and that the EDK has been downloaded to C:\EDK.

Create New Visual Studio Project
In this step, the goal is to set up Visual Studio so that it can build the Windows-hosted (NT32) UEFI environment. The build files for this project are located in C:\EDK\SAMPLE\PLATFORM\NT32.

1. Select File|New|Project.

2. Select the "General" project category and select "Makefile Project". Then enter the Name as "NT32". Then click "OK".

3. The Makefile Project Wizard will pop up. Click "Finish"

4. Your new project will appear in the Solution Explorer.

5. Select the NT32 project with the mouse. Select Project|Properties.

6. On the "NT32 Property Pages", change "Configuration" to "All Configurations".

7. Select "Build Command Line" and then select the ... on the right edge.

8. This will pop up a separate editor box. On three separate lines, enter the following text and click "OK".

cd /D c:\edk\sample\platform\nt32
set EDK_SOURCE=c:\edk
call build.bat

9. Now, for "Rebuild All Command Line" enter:

cd /D c:\edk\sample\platform\nt32

set EDK_SOURCE=c:\edk
call build.bat clean
call build.bat

10. Now, for "Clean Command Line" enter:

cd /D c:\edk\sample\platform\nt32

set EDK_SOURCE=c:\edk
call build.bat clean

11. Finally, for the last step, enter:

c:\edk\sample\platform\nt32\uefi\ia32\SecMain.exe

12. Select OK.












13. Edit Config.env, which is located in C:\EDK\Sample\Platform\NT32\Build, and change USE_VC8 = YES. By default, it is set to NO, which will build with Visual Studio 2003 and a large number of build errors.

Launching The Windows-Hosted Environment.
At this point, should be able to build by selecting Build|Build (or using F7). You can run the emulated environment by going to the command prompt and typing:

cd /d c:\edk\sample\platform\nt32\uefi\ia32
SecMain.exe

You will see (at least) two windows: the debug output console window and then the graphics output window. The graphics output window will show a fake logo, a progress bar and then boot into the built-in EFI Shell.



Conclusion
Well, that gets us to the first step. Next time we'll discuss how to set up the debugger and how to make your code debugger friendly.

UEFI HII (Part 10): Images

Before starting, you should be aware that the term 'image' has two different meanings in UEFI. First, it refers to executables (EXE), such as drivers and applications, that get loaded into memory by the LoadImage() and StartImage() services. Second, it refers to rectangular, full-color images.

Support for these images started with the UEFI 2.0 specification, with the EFI_GRAPHICS_OUTPUT_PROTOCOL. This protocol provide device-independent services for moving bitmapped images back and forth from a video device. But, it provided no standard means for finding and managing these images.

Then the UEFI 2.1 specification introduced the HII Database. Each driver can install pre-packaged images into the database. Then the driver can manipulate or display them using the EFI_HII_IMAGE_PROTOCOL.

Image Attributes
Images have the following attributes:

• Identifier. Along with the database handle, this identifier uniquely specifies the image.
• Width. The number of pixels per image row.
• Height. The number of image rows.
• Transparent. Indicates whether the pure black pixels (RGB(0,0,0)) will be drawn transparently over the background.
• Bitmap. Image pixels are organized left-to-right and then top-to-bottom. Each image pixel consists of 32-bits. The first 8-bits in each pixel are the blue (0 = off, 255 = on), the next 8-bits are the green and the next 8-bits are the blue.

Image Services

The UEFI HII Image protocol provides two classes of services:  

1. 
   
   Get/Set. You can get an image from the HII database, change an image in the HII database and add a new image to the HII database.
2. 
   
   Draw. You can draw an image, either using an image identifier or from a raw bitmapped image. In both cases, the drawing can be done directly to the screen or into another image, with or without clipping.

Conclusion

Images are slowly being integrated into firmware as an integral part, simply because of user expectations and a desire for OEMs to stand out in a crowd. UEFI 2.1 provides built-in support for basic image manipulation services. UEFI 2.3 also added support for animations. In our next article, we will look at how UEFI stores different types of images in the HII packages.

UEFI HII (Part 9): Font Package Encoding

Ok, now into the gory details of how font packages are encoded. In many ways, font packages are like string packages except, instead of strings, we are talking about font glyphs. There is a font header, followed by font information and then a series of font blocks.

The glyph blocks represent the glyphs in sorted order, starting with the glyph for character value 0x0001 and incrementing from there. Each glyph block increments the character value by either 1 or N. There are five basic types of glyph blocks:

1. Glyphs. These actually contain glyph data. There are variants for a single glyph or multiple glyphs, ones which use default character cell information or their own character cell information. The current character value is incremented by either 1 (for single glyph versions) or N (for multiple glyph versions).
2. Duplicate. This block duplicates the glyph data for a character value that has already been processed.  The current character value is incremented by 1.
3. Skip. These blocks skip a specified number of character values. For example, there might be no glyphs for character values 0x2000-0x20FF, but then one more glyph for character value 0x2100. A Skip block leaves a gap. The current character value is incremented by the skip count specified.
4. Defaults. This block sets up the default character cell information for the subsequent glyph blocks which use defaults. This is useful for fonts where a lot of characters share a specific set of information, such as Courier or other fixed-format fonts.
5. End. This marks the end of the glyph block information.

The actual glyph data is packed with one bit per pixel of the glyph, but rows are rounded up to the nearest byte (8 pixel) boundary. For example, consider the letter 'A'.

In this case, the actual glyph data is 9 pixels wide and 11 pixels high. If optimally packed, this would lead to a storage size of 99 bits or 13 bytes (12 with 3 bits left over). However, when this was being discussed, some preliminary implementation data suggested that the glyphs were much easier to handle if each row was byte aligned (the blue shaded area in the diagram). Then, after testing with compression software (since we assumed that the fonts would be stored compressed) that the bit-packed data actually was larger after compression than the byte-packed data.

Conclusion

This wraps up the second on fonts. Next week, we'll dive into how HII stores and handles images.

Harnessing The UEFI Shell

Just a quick plug to note that a book co-authored by myself and three Intel engineers (Mike Rothman, Bob Hale and Vincent Zimmer) is finally available for sale via Amazon or through Intel Press. UPS says that my personal copy arrives next Wednesday. Yeah!

UEFI HII (Part 8): Proportional Fonts

The UEFI Specification provids services and storage for manipulating bitmap fonts as part of the Human Interface Infrastructure (HII). Last week, we looked at the Simple Font, which describes a means of storing a fixed-width font for Unicode character values. Each character in the Simple Font is either in an 8 x 19 (narrow) pixel or 16 x 19 (wide) pixel character cell.

Proportional fonts extend bitmapped font support in several significant ways:

1. There can be multiple fonts, each described by its name, height and style (bold, italics, underline, etc.).
2. Each character cell in the font can be a different width.

Fonts
So, for UEFI, a font is a logical grouping of glyphs identified by a name, a size and a style.

• Font Name. "The font name describes, in broad terms, the visual style of the font." How do you tell that a character is part of the Arial font? Or Times New Roman? Well, they tend to share certain visual characteristics, such as whether they have serifs, or whether they are rounded or angular or represent alphabetic characters or symbols. 
• Font Size. "The font size describes the maximum height of the character cell, in pixels. The standard font always has the font size of 19." Rather than describing characters in terms of "points", the UEFI specification describes them in terms of the height of the character cell, in pixels. The UEFI specification does not use points, because (a) it only uses bitmapped fonts and (b) although points technically refers to 1/72nd of an inch, it actually does little to describe how much screen space a font will take up (except in comparison with another point size for the same font).
• Font Style. "The font style describes standard visual modifies to the base visual style of a font." In other words, you can have Arial, Arial + Bold, Arial + Italic, Arial + Bold and Italic, etc. They are still an Arial font, but they have additional stylistic modifiers. The UEFI specification explicitly supports bold, italic, underline, double-underline, embossed, outline and shadowed.

Glyphs
The glyph is the image representation of the character, with each dot in the image represented by a single bit, on or off. The off pixels are either not drawn (for transparent drawing) or else drawn in the background color. The on pixels are drawn in the foreground color. For example, for the letter A, it might look like this:

But this isn't enough information to draw the glyph. For one thing, we don't know how much space to leave between this glyph and other glyphs that we have drawn. This is one of the key differences between this font and the simple font in the previous articles. Simple fonts have the empty space above, below, before and after the character image built into the glyph. Proportional fonts track this sort of information separately, using three key ideas:

1. The character cell.
2. The baseline.
3. The character advance.

Character Cell

The character cell is a box that the glyph's image is positioned inside of. Each font has a character cell and each character glyph's image is placed relative to the top-left corner of that cell, or box. For example:

Baseline
The baseline (or Offset Y) allows UEFI firmware to draw text from different fonts on the same line and still have everything line up correctly. The baseline is calculated as the number of pixels above the Origin of the tallest character image.

 

In this picture, the baseline is the height of the tallest character (A). Notice that the character 'q' starts 3 pixels below the baseline, while both the 'A' and 'x' start on the baseline and the quotation mark (") starts 8 pixels above the baseline. The gap between the Origin and the left-most pixel of the glyph is known as Offset X. For the 'A' and 'q' and quotation mark, this is 1 pixel. For the 'x', this is 0 pixels.

Character Advance

The character advance is the number of pixels to move the Origin to the right. The character advance determines how much whitespace to put to the left and right of the glyph.

Conclusion

All of these details about offsets and baselines, advances and cells may seem intimidating, especially in a firmware environment. Many basic tasks become a lot more difficult when using proportional fonts: drawing text, editing a string or selecting text (with a mouse!). Fortunately, the EFI_HII_FONT_PROTOCOL provides functions which not only draw a string to a buffer or to the screen, but also report back information about the width and height of each character drawn. We will investigate these in more detail later.

Next week we will return to how the fonts are encoded.

UEFI HII (Part 7): Character Encoding

How do you know that the character 'A' corresponds to the character value 0x0041? Or that character value 0x215d is the character '⅝'? Well, standard, such as ASCII or the Unicode standard (or, going further back EBCDIC) describe the mapping between a numeric value and a specific characer.

But how to convert those numeric values into actual bits and bytes? 7-bits, 8-bits, 16-bits, 32-bits? And if more than one byte, is it big-endian or little-endian? That conversion process is called encoding.

I have the Unicode 1.0 specification sitting on my shelf. At that time, there was some optimism that all the character values that anyone would ever need could be contained in 16-bits: 65,536 character values. But even then, there were some signs that people were inventing and had invented many more character glyphs than could be contained. For example, in some scripts, individual cities had their own glyphs. And what about the Mahjong tile characters (0x1F000-0x1F002b)? So, prior to the Unicode 2.0 specification, the predominant form of encoding (known as UCS-2) embodied this 16-bit assumption.

But prior to that point, many operating systems (such as Windows NT) and firmware specifications (such as EFI 1.10) had taken root. While most operating systems have since migrated to the preferred encoding standard (UTF-16), which can handle the full set of Unicode character values, the UEFI specification still retains UCS-2.

So what is the big difference between UCS-2 and UTF-16? They are both 16-bit encoding schemes. For all the character values we care about, they are identical. Most of these characterThe real difference comes in how they handle character values beyond 0x10000 (that is, beyond character 65,536).

For UCS-2, these characters don't exist. There is no proper way to encode charcter 0x10001.  For UTF-16, characters values more than 0x10000 are encoded by combining two adjacent 16-bit characters. No one was particularly happy about this solution, since it took everyone back into the multi-byte character encoding nightmare that had plagued so many previous standards attempts (Shift-JI1, GB5, Windows code pages, etc.). But the alternative was to require a 32-bit unsigned integer to represent the characters, which would add a lot of bloat. But the advantage of this technique was that only a limit range of character values could appear as the first half of a surrogate pair (0xD800-0xDBFF) and another limited range of character values could appear as the second half (0xDC00-0xDFFF).  To calculate a character value, you take 6 bits from each and add 0x10000 to the result. This gives you a range of possible character values between 0x0000000-0xFFFFFFF. By separating the surrogate pair values in to first-half and second-half allowed string processing functions to work no matter where in the string they started from and which direction they processed the string. For more information, you can read the FAQ at the Unicode site.

But, for the purposes of the UEFI specification, those surrogate pair character values have no special meaning and would each be treated as a separate character. That is, 0xD800 0xDC00 would be a single character in UTF-16, but two characters under UEFI and UCS-2.

The UEFI specification does use another range of characters with special meaning defined in the Unicode specification: the Private Usage Areas. The Private Usage Area, which is a range of character values from 0xE000-0xF7FF) is left open to use by an application. In UEFI, these are used for embedding font control information directly into strings. The values 0xF620-0xF62B control turning specific font styles, such as bold, italics, etc. The values 0xF700-0xF7FF are used to select a specific font. The values 0xF800-0xF8FF select a font of a specific cell height. These values will never occur in normal text.

For more information on these character values, as well as others given special treatment in UEFI, see section 28.2.6.2 of the UEFI 2.3 specification.

Next time we'll continue by looking at the characteristics of Proportional Fonts.

UEFI HII (Part 6): Simple Fonts

Prior to to UEFI 2.10, there was only one graphics display mode: 800 x 600 x 32-bit color depth. Those numbers still show up as a required mode for external plug-in graphics adapters. But what if you want to draw text on those graphical displays? Where does the font come from? How big is it? Does the font really hold all 40K+ characters from the Unicode specification?

According to the EFI (and then UEFI), the system firmware was required to carry the ISO Latin-1 charcters (essentially characters U-0020 through U-00FF in the Unicode specification) as well as the line drawing characters. But that still leaves the question: how big are the bitmaps (or glyphs) in the font?

For a long time, prior to the UEFI 2.10 specification, the answer was 8 pixels wide and 19 pixels high. With a little math, you find that this works perfectly to put 80 columns and 25 rows of text on a 640 x 480 pixel display. That number of pixels, in turn, happens to be the highest resolution that could be produced by a standard VGA adapter. However, a character cell of 8 x 19 is not large enough to hold many of the glyphs found in other scripts, most notably Chinese, Japanese and Korean. So, in addition to the narrow 8 x 19 pixel character cells, a wide 16 x 19 pixel character cell was also supported.

One of the design goals for HII, back when UEFI 2.10 was being crafted, was the support for bitmapped, proportional fonts. But rather than throw away the previous way that font glyph encoding, the previous method was added as another package type: Simple Fonts (type 0x07).

Glyphs

Narrow glyphs consist of a header, followed by 19 bytes. Each of the bytes represents one row of the glyph. Byte 0 represents the top of the glyph and byte 18 represents the bottom of the glyph. Each bit in those bytes represents a single pixel, with a value of zero being off and a value of one being on. Bit 0 is the left-most pixel and bit 7 is the right-most pixel.

Wide glyphs consist of a header, followed by 38 bytes. Essentially they are encoded as two narrow glyphs, first the left half 8 x 19 image, followed by the right half 8 x 19 image.

Packages
The Simple Font package consists of the standard package header, a bit of housekeeping data and then an array of narrow glyph information, followed by an array of wide glyph information.
Each of the narrow glyphs has a small header, followed by the encoded glyph data.

typedef struct _EFI_HII_SIMPLE_FONT_PACKAGE_HDR {

  EFI_HII_PACKAGE_HEADER Header;

      UINT16           NumberOfNarrowGlyphs;
      UINT16           NumberOfWideGlyphs;
      EFI_NARROW_GLYPH NarrowGlyphs[];
      EFI_WIDE_GLYPH   WideGlyphs[];
    } EFI_HII_SIMPLE_FONT_PACKAGE_HDR;

Next week, we'll look at normal fonts.

typedef struct {

  CHAR16 UnicodeWeight;
  UINT8  Attributes;
  UINT8  GlyphCol1[EFI_GLYPH_HEIGHT];
} EFI_NARROW_GLYPH;

The Attributes indicate whether the glyph is narrow or wide and whether the character value is non-spacing. Normally, after drawing a character, the position of the next character would be directly to the right. For non-spacing characters, the next character would be drawn in exactly the same character cell. This allows, for example, the ` or ~ character to act as combining diacritical marks so that, when printed prior to an a or n, would display as á or ñ.
Each of the wide glyphs has a small header followed by the encoded glyph data:

typedef struct {

  CHAR16 UnicodeWeight;
  UINT8 Attributes;
  UINT8 GlyphCol1[EFI_GLYPH_HEIGHT];
  UINT8 GlyphCol2[EFI_GLYPH_HEIGHT];
  UINT8 Pad[3];
} EFI_WIDE_GLYPH;

Multiple Language Support
Let's say I'm designing a plug-in card and I want to support Chinese. But I don't know whether or not the system firmware in the platform where my card is installed supports all the Chinese characters (glyphs) that I need. What can I do? Well, with UEFI 2.10, your driver can just carry the characters you need that are in addition to those provided by the ISO Latin-1 characters that the platform is required to carry in its firmware. The HII Database will automatically combine glyphs from fonts which have the same font family, size and style.
Using The Simple Font
The simple font can be accessed, like all other fonts, through the HII Font protocol. The simple font has the name 'system' and has a height of 19 pixels.

Harnessing The UEFI Shell Book

Ok, a little promotion for the book I co-wrote, which is finally for sale. We finished writing it over 6 months ago. Not that most of you are going to run out and write UEFI Shell apps. But if you were...


http://www.intel.com/intelpress/sum_eshl.htm

UEFI HII (Part 5): Strings API

Last time we learned that strings in a specific language are grouped together in packages and packages are grouped together in package lists. Strings not only have text and a language, but they also have an associated font, font size and font style.

To get to a specific string, you need three things: the package list handle (EFI_HII_HANDLE), the string identifier (EFI_STRING_ID) and the language. Since you are normally using the platform's current language, you generally don't have to worry about that and use the default setting.

There are two sets of string-related protocols: (a) those which add, modify and remove strings from the HII Database and (b) those which create a bitmap using the strings text.

With EFI_HII_STRING_PROTOCOL, you can create new strings, retrieve the string's text, change the string's text and find out which languages are supported.

• NewString() lets you create a new string within a specific package list and returns back a new string identifier. You can specify (if you want), the language and the font information. If you don't specify the language, the platform's current language is used. If you don't specify a font, then the platform's standard font is used.
• GetString() retrieves information about a specific string, including the string's text and the associated font. If you don't specify the language, the platform's current language is used.
• SetString() changes information about a specific string, including the string's text and (optionally), the string's font information. If you don't specify the language, the platform's current language is used.
• GetLanguages() reports the languages that are supported by a specific package list.
• GetSecondaryLanguages() reports the regional versions of a specific language supported by a specific package list. For example, if you passed in "en", it might return "en-US", "en-UK" "en-PH" if there were specific translations for those regional variations in English.

The EFI_HII_FONT_PROTOCOL deals with fonts and we'll return to that subject in a later article, in all its proportional font glory. Right now, we'll just focus on the two functions which deal with strings:

• StringToImage() draws a string's text onto a bitmap or the screen at a specific location. If the font is not specified, then the platform's standard font is used instead. If the language is not specified, then the platform's current language is used.  This function also allows you to clip the text to a rectangle in a variety of ways, wrap the text, draw transparently and handle multi-line text. Even more useful to sophisticated users, it also reports the positioning of each character in the string in the bitmap so that user-interface code can use it for cursor movement and mouse selection.
• StringIdToImage() does the same thing, except that you pass in a string identifier and a package list handle.

Next time we'll start looking into the wonderful world of bitmapped fonts.

Tim

UEFI HII (Part 4): Strings

Up to this point, we've discussed the HII Database and how individual drivers can contribute resources (strings, fonts, images, forms, etc.) in the form of packages to that database. Groups of packages are called package lists. Later the form browser extracts these package lists from the database to use in constructing the user interface for platform configuration and other user-interface tasks.

One type of package that drivers can contribute to the HII Database is the strings package. A string package is a collection of strings associated with a specific language. Each string has a number (called the string identifier) that uniquely identifies it within the package list, and default font information, such as font name, size and style.

Languages.
Each string package only contains text in one language. For example, one string package contains English (U.S.), another Korean, another Pilipino and another French. By separating the languages into separate string packages, it is easy to delete support for a particular language: just delete the associated package.

The UEFI firmware maintains the system user-interface language information in two special EFI variables:

1. PlatformLangCodes. The list of languages that the platform supports.
2. PlatformLang. The current platform language.

These EFI variables are also used by other UEFI protocols, such as the EFI Driver Configuration protocol, the EFI Driver Diagnostics protocol, the Component Name protocol and the Unicode Collation protocol.

Languages are encoded according to RFC 4646, which specifies two and three letter codes for each language, along with additional modifiers representing a specific geographic location. For example:

• en-US (English, United States)
• fr (French)
• fr-FR (French, France)
• zh-CN (Chinese, mainland China)
• sr-CS (Serbian, Serbia/Montenegro)

The rules by which a form browser might substitute an alternate language (say, Portuguese from Portugal if there was no Portuguese from Brazil, or English if there was no French), is specific to the implementation.

It is possible to find out which languages are supported by iterating through all of the package lists in the system (using ListPackageLists()) and then using GetLanguages() and GetSecondaryLanguages().

Fonts
Each string is associated with a specific font family, font size and font style. The form browser may choose to use this information, ignore it completely or substitute a similar but different font. So this font information might be considered more of a suggestion, rather than a command. In many cases, firmware implementations may use this information to cull the fonts that are included in the BIOS ROM (for space reasons) so that the font only contains the characters used.

The HII-related protocols (such as HII Font or HII String) will use the font information associated with the string to select the display font unless an alternate is provided by the caller.

There are three font attributes associated with each string.

• Font Name. The font name the family of font (Arial, Helvetica, Times Roman, Courier), which identifies, in broad terms, the visual style of the font.  
• Font Size. This is the cell height, in pixels. To give some perspective, the "standard" UEFI font size is 19 pixels high.
• Font Style. The font style indicates how the basic font should be modified. The following styles can be described: bold, italic, emboss, outline, shadow, underline and double-underline.

If the form browser does not have access to the exact font specified by a string, it might substitute a different font or it might synthesize using an algorithm. An example of substitution would be using Helvetica instead of Arial. An example of sythesizing would be if a doubled 12-size font were used for a 24-size font or if the italic style were simulated by shifting the successive lines of a glyph over by one pixel so that it would slant.

Identifiers
Strings are uniquely identified in the system by a string identifier (EFI_STRING_ID), a package list handle (EFI_HII_HANDLE) and a language. If the system's display language is English ('en-US') and you ask for string #1, you would get string #1 from the English string package. If the system's display language is French ('fr') and you ask for string #1, you would get string #1 from the French string package. Likewise, for Japanese ('jp'). As a rule, UEFI drivers don't hand around pointers to null-terminated strings. Instead, they pass around string identifiers and package list handles.

Can you actually examine and modify the text? Of course. GetString() retrieves the actual text and SetString() lets you modify it. NewString() let's you create a brand new string, with a unique string identifier.

Encoding
Each string package begins with the standard header (EFI_HII_PACKAGE_HEADER) with the Type set to EFI_HII_PACKAGE_STRINGS. Following the standard header is the string-package-specific header:

typedef struct _EFI_HII_STRING_PACKAGE_HDR {

  EFI_HII_PACKAGE_HEADER Header;
  UINT32 HdrSize;
  UINT32 StringInfoOffset;
  CHAR16 LanguageWindow[16];
  EFI_STRING_ID LanguageName;
//CHAR8 Language[ … ];
} EFI_HII_STRING_PACKAGE_HDR; 

The actual string data begins at StringInfoOffset bytes from the start of this structure. The LanguageWindow array is used for setting up the default "windows" used for the compression algorithm. More on this later. Language is the RFC 4646 null-termined language string which identifies which language this package is for. For example, "en-US" or "fr" or "jp". LanguageName is the string identifier of the string that gives the user-readable name of the language this package is for. For example, "English" or "French" or "Japanese" These can be used when presenting choices to a user.

The string information consists of a series of records, which can be broken down into three categories:

1. String Records. These records assign the current string identifier value to specific string text .
2. Identifier Records. These records change the current string identifier value.
3. Font Records. These records describe the fonts used by later strings.

String Records
String records are broken down into three types:

1. Use compressed text or uncompressed text. Text can be compressed using the Standard Compression Scheme for Unicode (SCSU), which is described in Unicode's Technical Report #6. Optimized for reducing the number of bytes required to describe Korean, Japanese and Chinese characters, this scheme uses the concept of "windows" of 127 characters than can be selected for a sub-string of characters. The default settings for these windows are specified in the report. But they can also be optimized for the exact strings in the package by altering the values in the LanguageWindow array in the header. Uncompressed text is simply listed as a null-termined UCS-2 string.
2. List single string or multiple strings. Strings which have string identifiers which are sequential can be listed in a single record. Or a single string can be listed.
3. Use the default font or a specific font. Strings which use the default font require fewer bytes to encode because the font is implied.
4. Use text provided or duplicate text. One of the record types simply implies that the text for the new string is a copy of the text for a string which was previously defined.

As mentioned before, strings are associated with a specific font. However, the fonts can be changed in the middle of a string using a series of special control characters. The character values used are marked as "implementation-specific" in the Unicode specification:

For example, characters 0xF7xx (where xx is the font identifier assigned by a font record, below) can be used to switch fonts. Characters 0xF8xx (where xx is the font size) can be used to change just the font's size. Characters 0xF620 and 0xF621 turn bold on and off. Characters 0xF622 and 0xF623 turn italic on and off. Characters 0xF624 and 0xF625 turn underline on and off. Characters 0xF626 and 0xF627 turn emboss on and off. Characters 0xF628 and 0xF629 turn outline on and off. Characters 0xF62A and 0xF62B turn double underline on and off.

Identifier Records
Identifier records are used to adjust the current string identifier value without assigning any string text. This can be useful when there are gaps in the string identifiers. When processing the string records, the current string identifier is always set to 1 and is incremented each time a string record is processed. So, normally, the first string is assigned identifier #1, the second #2, etc. But if the identifiers are not sequential (i.e. 1,2, 16) you can use a skip record so that after the second string, you just skip the next 13 identifiers.

Font Records
The font records must appear before the first instance of a string that uses them. The exception for this is the "default" font which is initialize to the system's default font. Each font is assigned a number that is only valid within the package, starting with 0 (for the default font) and going upwards. That means there is a theoretical maximum of 256 fonts used wihtin a string package. In addition to the identifier, each font has the usual attributes (name, size, style).

Conclusion
Strings are an important part of any user-interface. The ability and the flexibility to display strings in multiple languages, using a variety of font styles, sizes and families is important in making a rich user interface.

Next time, we begin to delve into the wicked world of HII fonts.

UEFI @ Intel IDF.

I just saw that they posted publicly the information from Intel's IDF, including the UEFI track, here. You can see a number of interesting articles from the industry heavyweights, like Dell, IBM, Microsoft and Intel. Not to mention the co-authors of a book (Vincent Zimmer, Mike Rothman) about the UEFI Shell. Good reading.

UEFI HII (Part 3)

The UEFI Human Interface Infrastructure (or HII) provides a means for drivers provided by 3rd party hardware and software vendors to expose their configuration settings. Then, a browser or provisioning application can store, restore or change those configuration settings. The configuration settings are encoded as packages. There are several different types of packages defined in the UEFI specification: fonts, strings, images, animations and, most importantly, forms. Each package has the following header:

typedef struct {
  UINT32 Length:24;
  UINT32 Type:8;
//UINT8 Data[…];
} EFI_HII_PACKAGE_HEADER;

This structure contains both the package Type (font, form, image, etc.) and the package Length (including the header), in bytes. Each driver can group these packages (called a package list) and install them in the HII Database using NewPackageList(). Then later an application can find them and display them. The package list has the following header:

typedef struct {
  EFI_GUID PackageListGuid;
  UINT32 PackagLength;
} EFI_HII_PACKAGE_LIST_HEADER;

So that leaves two big questions:

1. How do you create the packages and package lists?
2. How does the driver find them?

Well, the first question is a bit tricky and each BIOS vendor probably answers this question differently. The EDK (published by Intel at http://www.tianocore.org/), uses a script language called Visual Forms Representation (or VFR) that is compiled down into IFR. It also specifies strings in special .UNI files, which associate a label with a string and language. At the end of the day, these different package types get built into a binary. And that binary can be packaged four ways:

1. Built-Into The Driver. In this method, a tool takes the bytes that make up the package list and generates an assembly-language (.ASM) source file, which is then linked together with the rest of the driver. This method makes it easy to find the package list, since it has a normal label that resolves during the linking process.
2. Separate File. With this method, the binary is packaged up as a normal firmware file with a special file type or, more commonly, a special file name. The driver then searches the firmware volume in which it resides for a file with the special name, loads it into memory and then gets a pointer to the first byte. This method takes a little more work, but it allows the package list to be generated separately. Since the package list can be easily located, it can then be edited at some point after the driver has been compiled but before the final flash image is created. This is very useful when you want to process the package lists without ever touching the source code. For example, if your driver ships supporting 30 languages, but you only have ROM space for 3, you could either recompile the driver or you could just edit the binary information. Or perhaps you want to allow a downstream VAR to substitute their logo for the generic logo. By making the binary form of the package list easy to locate, these changes can be made easily. This method also allows files to be on separate media, such as a disk.
3. Same File, Separate Section. This method is similar to the method described above, in that the binary information is packaged separately from the EXE. However, here, the binary is included as a separate section in the same file. The Firmware File Specification (either the older Intel Tiano version or the newer UEFI PI version) allows certain file types to be broken up into sections. One section contains the driver itself and, in this case, another section contains the package list. This shares many of the advantages of the previous method, but creates a direct association between the driver and its related forms, fonts, strings and images.
4. Same File, Resources. This method embeds the binary into the EXE portion of the file as a resource with the resource type "HII" (see the LoadImage()). When LoadImage() runs, it looks for the resource and, if its is found, installs a protocol on image's handle with the GUID EFI_HII_PACKAGE_LIST_PROTOCOL_GUID that contains a pointer to the package list. A driver uses HandleProtocol on its own image handle to find a pointer. This has an advantage over the previous methods in that it does not require a UEFI PI image or the firmware file system. So therefore it is suitable for pure UEFI drivers or drivers that are loaded from a plug-in card's option ROM.

After finding the resources, the driver simply passes a pointer to the package list into NewPackageList().

Next time we'll look in greater detail at the different types of packages, starting with the strings.

Phoenix Demos 1 Second UEFI Boot

Several news sources have posted coverage of IDF 2009, where Phoenix (that's who I work for) booting a UEFI-based system in 1+ seconds and Windows 7 in 7-10 seconds. You can read about it (for example) here.

The UEFI Shell: Moving The Platform Beyond DOS

Hey, just a plug for a book that I co-wrote with three other UEFI experts (Mike Rothman, Vincent Zimmer and Bob Hale) about the UEFI Shell. You can read an excerpt from the book on the Intel Press web site here.