Edit
kc3-lang/harfbuzz/docs/usermanual-integration.xml
n8willis
- docs/usermanual-integration.xml
-
<?xml version="1.0"?> <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.3//EN" "http://www.oasis-open.org/docbook/xml/4.3/docbookx.dtd" [ <!ENTITY % local.common.attrib "xmlns:xi CDATA #FIXED 'http://www.w3.org/2003/XInclude'"> <!ENTITY version SYSTEM "version.xml"> ]> <chapter id="integration"> <title>Platform Integration Guide</title> <para> HarfBuzz was first developed for use with the GNOME and GTK software stack commonly found in desktop Linux distributions. Nevertheless, it can be used on other operating systems and platforms, from iOS and macOS to Windows. It can also be used with other application frameworks and components, such as Android, Qt, or application-specific widget libraries. </para> <para> This chapter will look at how HarfBuzz fits into a typical text-rendering pipeline, and will discuss the APIs available to integrate HarfBuzz with contemporary Linux, Mac, and Windows software. It will also show how HarfBuzz integrates with popular external libraries like FreeType and International Components for Unicode (ICU) and describe the HarfBuzz language bindings for Python. </para> <para> On a GNOME system, HarfBuzz is designed to tie in with several other common system libraries. The most common architecture uses Pango at the layer directly "above" HarfBuzz; Pango is responsible for text segmentation and for ensuring that each input <type>hb_buffer_t</type> passed to HarfBuzz for shaping contains Unicode code points that share the same segment properties (namely, direction, language, and script, but also higher-level properties like the active font, font style, and so on). </para> <para> The layer directly "below" HarfBuzz is typically FreeType, which is used to rasterize glyph outlines at the necessary optical size, hinting settings, and pixel resolution. FreeType provides APIs for accessing font and face information, so HarfBuzz includes functions to create <type>hb_face_t</type> and <type>hb_font_t</type> objects directly from FreeType objects. HarfBuzz can use FreeType's built-in functions for <structfield>font_funcs</structfield> vtable in an <type>hb_font_t</type>. </para> <para> FreeType's output is bitmaps of the rasterized glyphs; on a typical Linux system these will then be drawn by a graphics library like Cairo, but those details are beyond HarfBuzz's control. On the other hand, at the top end of the stack, Pango is part of the larger GNOME framework, and HarfBuzz does include APIs for working with key components of GNOME's higher-level libraries — most notably GLib. </para> <para> For other operating systems or application frameworks, the critical integration points are where HarfBuzz gets font and face information about the font used for shaping and where HarfBuzz gets Unicode data about the input-buffer code points. </para> <para> The font and face information is necessary for text shaping because HarfBuzz needs to retrieve the glyph indices for particular code points, and to know the extents and advances of glyphs. Note that, in an OpenType variable font, both of those types of information can change with different variation-axis settings. </para> <para> The Unicode information is necessary for shaping because the properties of a code point (such as its General Category (gc), Canonical Combining Class (ccc), and decomposition) can directly impact the shaping moves that HarfBuzz performs. </para> <section id="integration-glib"> <title>GNOME integration, GLib, and GObject</title> <para> As mentioned in the preceding section, HarfBuzz offers integration APIs to help client programs using the GNOME and GTK framework commonly found in desktop Linux distributions. </para> <para> GLib is the main utility library for GNOME applications. It provides basic data types and conversions, file abstractions, string manipulation, and macros, as well as facilities like memory allocation and the main event loop. </para> <para> Where text shaping is concerned, GLib provides several utilities that HarfBuzz can take advantage of, including a set of Unicode-data functions and a data type for script information. Both are useful when working with HarfBuzz buffers. To make use of them, you will need to include the <filename>hb-glib.h</filename> header file. </para> <para> GLib's <ulink url="https://developer.gnome.org/glib/stable/glib-Unicode-Manipulation.html">Unicode manipulation API</ulink> includes all the functionality necessary to retrieve Unicode data for the <structfield>unicode_funcs</structfield> structure of a HarfBuzz <type>hb_buffer_t</type>. </para> <para> The function <function>hb_glib_get_unicode_funcs()</function> sets up a <type>hb_unicode_funcs_t</type> structure configured with the GLib Unicode functions and returns a pointer to it. </para> <para> You can attach this Unicode-functions structure to your buffer, and it will be ready for use with GLib: </para> <programlisting language="C"> #include <hb-glib.h> ... hb_unicode_funcs_t *glibufunctions; glibufunctions = hb_glib_get_unicode_funcs(); hb_buffer_set_unicode_funcs(buf, glibufunctions); </programlisting> <para> For script information, GLib uses the <type>GUnicodeScript</type> type. Like HarfBuzz's own <type>hb_script_t</type>, this data type is an enumeration of Unicode scripts, but text segments passed in from GLib code will be tagged with a <type>GUnicodeScript</type>. Therefore, when setting the script property on a <type>hb_buffer_t</type>, you will need to convert between the <type>GUnicodeScript</type> of the input provided by GLib and HarfBuzz's <type>hb_script_t</type> type. </para> <para> The <function>hb_glib_script_to_script()</function> function takes an <type>GUnicodeScript</type> script identifier as its sole argument and returns the corresponding <type>hb_script_t</type>. The <function>hb_glib_script_from_script()</function> does the reverse, taking an <type>hb_script_t</type> and returning the <type>GUnicodeScript</type> identifier for GLib. </para> <para> Finally, GLib also provides a reference-counted object type called <ulink url="https://developer.gnome.org/glib/stable/glib-Byte-Arrays.html#GBytes"><type>GBytes</type></ulink> that is used for accessing raw memory segments with the benefits of GLib's lifecycle management. HarfBuzz provides a <function>hb_glib_blob_create()</function> function that lets you create an <type>hb_blob_t</type> directly from a <type>GBytes</type> object. This function takes only the <type>GBytes</type> object as its input; HarfBuzz registers the GLib <function>destroy</function> callback automatically. </para> <para> The GNOME platform also features an object system called GObject. For HarfBuzz, the main advantage of GObject is a feature called <ulink url="https://gi.readthedocs.io/en/latest/">GObject Introspection</ulink>. This is a middleware facility that can be used to generate language bindings for C libraries. HarfBuzz uses it to build its Python bindings, which we will look at in a separate section. </para> </section> <section id="integration-freetype"> <title>FreeType integration</title> <para> FreeType is the free-software font-rendering engine included in desktop Linux distributions, Android, ChromeOS, iOS, and multiple Unix operating systems, and used by cross-platform programs like Chrome, Java, and GhostScript. Used together, HarfBuzz can perform shaping on Unicode text segments, outputting the glyph IDs that FreeType should rasterize from the active font as well as the positions at which those glyphs should be drawn. </para> <para> HarfBuzz provides integration points with FreeType at the face-object and font-object level and for the font-functions virtual-method structure of a font object. To use the FreeType-integration API, include the <filename>hb-ft.h</filename> header. </para> <para> In a typical client program, you will create your <type>hb_face_t</type> face object and <type>hb_font_t</type> font object from a FreeType <type>FT_Face</type>. HarfBuzz provides a suite of functions for doing this. </para> <para> In the most common case, you will want to use <function>hb_ft_font_create_referenced()</function>, which creates both an <type>hb_face_t</type> face object and <type>hb_font_t</type> font object (linked to that face object), and provides lifecycle management. </para> <para> It is important to note, though, that while HarfBuzz makes a distinction between its face and font objects, FreeType's <type>FT_Face</type> does not. After you create your <type>FT_Face</type>, you must set its size parameter using <function>FT_Set_Char_Size()</function>, because an <type>hb_font_t</type> is defined as an instance of an <type>hb_face_t</type> with size specified. </para> <programlisting language="C"> #include <hb-ft.h> ... FT_New_Face(ft_library, font_path, index, &face); FT_Set_Char_Size(face, 0, 1000, 0, 0); hb_font_t *font = hb_ft_font_create(face); </programlisting> <para> <function>hb_ft_font_create_referenced()</function> is the recommended function for creating an <type>hb_face_t</type> face object. This function calls <function>FT_Reference_Face()</function> before using the <type>FT_Face</type> and calls <function>FT_Done_Face()</function> when it is finished using the <type>FT_Face</type>. Consequently, your client program does not need to worry about destroying the <type>FT_Face</type> while HarfBuzz is still using it. </para> <para> Although <function>hb_ft_font_create_referenced()</function> is the recommended function, there is another variant for client code where special circumstances make it necessary. The simpler version of the function is <function>hb_ft_font_create()</function>, which takes an <type>FT_Face</type> and an optional destroy callback as its arguments. Because <function>hb_ft_font_create()</function> does not offer lifecycle management, however, your client code will be responsible for tracking references to the <type>FT_Face</type> objects and destroying them when they are no longer needed. If you do not have a valid reason for doing this, use <function>hb_ft_font_create_referenced()</function>. </para> <para> After you have created your font object from your <type>FT_Face</type>, you can set or retrieve the <structfield>load_flags</structfield> of the <type>FT_Face</type> through the <type>hb_font_t</type> object. HarfBuzz provides <function>hb_ft_font_set_load_flags()</function> and <function>hb_ft_font_get_load_flags()</function> for this purpose. The ability to set the <structfield>load_flags</structfield> through the font object could be useful for enabling or disabling hinting, for example, or to activate vertical layout. </para> <para> HarfBuzz also provides a utility function called <function>hb_ft_font_has_changed()</function> that you should call whenever you have altered the properties of your underlying <type>FT_Face</type>, as well as a <function>hb_ft_get_face()</function> that you can call on an <type>hb_font_t</type> font object to fetch its underlying <type>FT_Face</type>. </para> <para> With an <type>hb_face_t</type> and <type>hb_font_t</type> both linked to your <type>FT_Face</type>, you will typically also want to use FreeType for the <structfield>font_funcs</structfield> vtable of your <type>hb_font_t</type>. As a reminder, this font-functions structure is the set of methods that HarfBuzz will use to fetch important information from the font, such as the advances and extents of individual glyphs. </para> <para> All you need to do is call </para> <programlisting language="C"> hb_ft_font_set_funcs(font); </programlisting> <para> and HarfBuzz will use FreeType for the font-functions in <literal>font</literal>. </para> <para> As we noted above, an <type>hb_font_t</type> is derived from an <type>hb_face_t</type> with size (and, perhaps, other parameters, such as variation-axis coordinates) specified. Consequently, you can reuse an <type>hb_face_t</type> with several <type>hb_font_t</type> objects, and HarfBuzz provides functions to simplify this. </para> <para> The <function>hb_ft_face_create_referenced()</function> function creates just an <type>hb_face_t</type> from a FreeType <type>FT_Face</type> and, as with <function>hb_ft_font_create_referenced()</function> above, provides lifecycle management for the <type>FT_Face</type>. </para> <para> Similarly, there is an <function>hb_ft_face_create()</function> function variant that does not provide the lifecycle-management feature. As with the font-object case, if you use this version of the function, it will be your client code's respsonsibility to track usage of the <type>FT_Face</type> objects. </para> <para> A third variant of this function is <function>hb_ft_face_create_cached()</function>, which is the same as <function>hb_ft_face_create()</function> except that it also uses the <structfield>generic</structfield> field of the <type>FT_Face</type> structure to save a pointer to the newly created <type>hb_face_t</type>. Subsequently, function calls that pass the same <type>FT_Face</type> will get the same <type>hb_face_t</type> returned — and the <type>hb_face_t</type> will be correctly reference counted. Still, as with <function>hb_ft_face_create()</function>, your client code must track references to the <type>FT_Face</type> itself, and destroy it when it is unneeded. </para> </section> <section id="integration-uniscribe"> <title>Uniscribe integration</title> <para> If your client program is running on Windows, HarfBuzz offers an additional API that can help integrate with Microsoft's Uniscribe engine and the Windows GDI. </para> <para> Overall, the Uniscribe API covers a broader set of typographic layout functions than HarfBuzz implements, but HarfBuzz's shaping API can serve as a drop-in replacement for Uniscribe's shaping functionality. In fact, one of HarfBuzz's design goals is to accurately reproduce the same output for shaping a given text segment that Uniscribe produces — even to the point of duplicating known shaping bugs or deviations from the specification — so you can be confident that your users' documents with their existing fonts will not be affected adversely by switching to HarfBuzz. </para> <para> At a basic level, HarfBuzz's <function>hb_shape()</function> function replaces both the <ulink url=""><function>ScriptShape()</function></ulink> and <ulink url="https://docs.microsoft.com/en-us/windows/desktop/api/Usp10/nf-usp10-scriptplace"><function>ScriptPlace()</function></ulink> functions from Uniscribe. </para> <para> However, whereas <function>ScriptShape()</function> returns the glyphs and clusters for a shaped sequence and <function>ScriptPlace()</function> returns the advances and offsets for those glyphs, <function>hb_shape()</function> handles both. After <function>hb_shape()</function> shapes a buffer, the output glyph IDs and cluster IDs are returned as an array of <structname>hb_glyph_info_t</structname> structures, and the glyph advances and offsets are returned as an array of <structname>hb_glyph_position_t</structname> structures. </para> <para> Your client program only needs to ensure that it coverts correctly between HarfBuzz's low-level data types (such as <type>hb_position_t</type>) and Windows's corresponding types (such as <type>GOFFSET</type> and <type>ABC</type>). Be sure you read the <xref linkend="buffers-language-script-and-direction" /> chapter for a full explanation of how HarfBuzz input buffers are used, and see <xref linkend="shaping-buffer-output" /> for the details of what <function>hb_shape()</function> returns in the output buffer when shaping is complete. </para> <para> Although <function>hb_shape()</function> itself is functionally equivalent to Uniscribe's shaping routines, there are two additional HarfBuzz functions you may want to use to integrate the libraries in your code. Both are used to link HarfBuzz font objects to the equivalent Windows structures. </para> <para> The <function>hb_uniscribe_font_get_logfontw()</function> function takes a <type>hb_font_t</type> font object and returns a pointer to the <ulink url="https://docs.microsoft.com/en-us/windows/desktop/api/wingdi/ns-wingdi-logfontw"><type>LOGFONTW</type></ulink> "logical font" that corresponds to it. A <type>LOGFONTW</type> structure holds font-wide attributes, including metrics, size, and style information. </para> <!-- <para> In Uniscribe's model, the <type>SCRIPT_CACHE</type> holds the device context, including the logical font that the shaping functions apply. https://docs.microsoft.com/en-us/windows/desktop/Intl/script-cache </para> --> <para> The <function>hb_uniscribe_font_get_hfont()</function> function also takes a <type>hb_font_t</type> font object, but it returns an <type>HFONT</type> — a handle to the underlying logical font — instead. </para> <para> <type>LOGFONTW</type>s and <type>HFONT</type>s are both needed by other Uniscribe functions. </para> <para> As a final note, you may notice a reference to an optional <literal>uniscribe</literal> shaper back-end in the <xref linkend="configuration" /> section of the HarfBuzz manual. This option is not a Uniscribe-integration facility. </para> <para> Instead, it is a internal code path used in the <command>hb-shape</command> command-line utility, which hands shaping functionality over to Uniscribe entirely, when run on a Windows system. That allows testing HarfBuzz's native output against the Uniscribe engine, for tracking compatibility and debugging. </para> <para> Because this back-end is only used when testing HarfBuzz functionality, it is disabled by default when building the HarfBuzz binaries. </para> </section> <section id="integration-coretext"> <title>Core Text integration</title> <para> If your client program is running on macOS or iOS, HarfBuzz offers an additional API that can help integrate with Apple's Core Text engine and the underlying Core Graphics framework. HarfBuzz does not attempt to offer the same drop-in-replacement functionality for Core Text that it strives for with Uniscribe on Windows, but you can still use HarfBuzz to perform text shaping in native macOS and iOS applications. </para> <para> Note, though, that if your interest is just in using fonts that contain Apple Advanced Typography (AAT) features, then you do not need to add Core Text integration. HarfBuzz natively supports AAT features and will shape AAT fonts (on any platform) automatically, without requiring additional work on your part. This includes support for AAT-specific TrueType tables such as <literal>mort</literal>, <literal>morx</literal>, and <literal>kerx</literal>, which AAT fonts use instead of <literal>GSUB</literal> and <literal>GPOS</literal>. </para> <para> On a macOS or iOS system, the primary integration points offered by HarfBuzz are for face objects and font objects. </para> <para> The Apple APIs offer a pair of data structures that map well to HarfBuzz's face and font objects. The Core Graphics API, which is slightly lower-level than Core Text, provides <ulink url="https://developer.apple.com/documentation/coregraphics/cgfontref"><type>CGFontRef</type></ulink>, which enables access to typeface properties, but does not include size information. Core Text's <ulink url="https://developer.apple.com/documentation/coretext/ctfont-q6r"><type>CTFontRef</type></ulink> is analagous to a HarfBuzz font object, with all of the properties required to render text at a specific size and configuration. Consequently, a HarfBuzz <type>hb_font_t</type> font object can be hooked up to a Core Text <type>CTFontRef</type>, and a HarfBuzz <type>hb_face_t</type> face object can be hooked up to a <type>CGFontRef</type>. </para> <para> You can create a <type>hb_face_t</type> from a <type>CGFontRef</type> by using the <function>hb_coretext_face_create()</function>. Subsequently, you can retrieve the <type>CGFontRef</type> from a <type>hb_face_t</type> with <function>hb_coretext_face_get_cg_font()</function>. </para> <para> Likewise, you create a <type>hb_font_t</type> from a <type>CTFontRef</type> by calling <function>hb_coretext_font_create()</function>, and you can fetch the associated <type>CTFontRef</type> from a <type>hb_font_t</type> font object with <function>hb_coretext_face_get_ct_font()</function>. </para> <para> HarfBuzz also offers a <function>hb_font_set_ptem()</function> that you an use to set the nominal point size on any <type>hb_font_t</type> font object. Core Text uses this value to implement optical scaling. </para> <para> When integrating your client code with Core Text, it is important to recognize that Core Text <literal>points</literal> are not typographic points (standardized at 72 per inch) as the term is used elsewhere in OpenType. Instead, Core Text points are CSS points, which are standardized at 96 per inch. </para> <para> HarfBuzz's font functions take this distinction into account, but it can be an easy detail to miss in cross-platform code. </para> <para> As a final note, you may notice a reference to an optional <literal>coretext</literal> shaper back-end in the <xref linkend="configuration" /> section of the HarfBuzz manual. This option is not a Core Text-integration facility. </para> <para> Instead, it is a internal code path used in the <command>hb-shape</command> command-line utility, which hands shaping functionality over to Core Text entirely, when run on a macOS system. That allows testing HarfBuzz's native output against the Core Text engine, for tracking compatibility and debugging. </para> <para> Because this back-end is only used when testing HarfBuzz functionality, it is disabled by default when building the HarfBuzz binaries. </para> </section> <section id="integration-icu"> <title>ICU integration</title> <para> Although HarfBuzz includes its own Unicode-data functions, it also provides integration APIs for using the International Components for Unicode (ICU) library as a source of Unicode data on any supported platform. </para> <para> The principal integration point with ICU is the <type>hb_unicode_funcs_t</type> Unicode-functions structure attached to a buffer. This structure holds the virtual methods used for retrieving Unicode character properties, such as General Category, Script, Combining Class, decomposition mappings, and mirroring information. </para> <para> To use ICU in your client program, you need to call <function>hb_icu_get_unicode_funcs()</function>, which creates a Unicode-functions structure populated with the ICU function for each included method. Subsequently, you can attach the Unicode-functions structure to your buffer: </para> <programlisting language="C"> hb_unicode_funcs_t *icufunctions; icufunctions = hb_icu_get_unicode_funcs(); hb_buffer_set_unicode_funcs(buf, icufunctions); </programlisting> <para> and ICU will be used for Unicode-data access. </para> <para> HarfBuzz also supplies a pair of functions (<function>hb_icu_script_from_script()</function> and <function>hb_icu_script_to_script()</function>) for converting between ICU's and HarfBuzz's internal enumerations of Unicode scripts. The <function>hb_icu_script_from_script()</function> function converts from a HarfBuzz <type>hb_script_t</type> to an ICU <type>UScriptCode</type>. The <function>hb_icu_script_to_script()</function> function does the reverse: converting from a <type>UScriptCode</type> identifier to a <type>hb_script_t</type>. </para> <para> By default, HarfBuzz's ICU support is built as a separate shared library (<filename class="libraryfile">libharfbuzz-icu.so</filename>) when compiling HarfBuzz from source. This allows client programs that do not need ICU to link against HarfBuzz without unnecessarily adding ICU as a dependency. You can also build HarfBuzz with ICU support built directly into the main HarfBuzz shared library (<filename class="libraryfile">libharfbuzz.so</filename>), by specifying the <literal>--with-icu=builtin</literal> compile-time option. </para> </section> <section id="integration-python"> <title>Python bindings</title> <para> As noted in the <xref linkend="integration-glib" /> section, HarfBuzz uses a feature called <ulink url="https://wiki.gnome.org/Projects/GObjectIntrospection">GObject Introspection</ulink> (GI) to provide bindings for Python. </para> <para> At compile time, the GI scanner analyzes the HarfBuzz C source and builds metadata objects connecting the language bindings to the C library. Your Python code can then use the HarfBuzz binary through its Python interface. </para> <para> HarfBuzz's Python bindings support Python 2 and Python 3. To use them, you will need to have the <literal>pygobject</literal> package installed. Then you should import <literal>HarfBuzz</literal> from <literal>gi.repository</literal>: </para> <programlisting language="Python"> from gi.repository import HarfBuzz </programlisting> <para> and you can call HarfBuzz functions from Python. Sample code can be found in the <filename>sample.py</filename> script in the HarfBuzz <filename>src</filename> directory. </para> <para> Do note, however, that the Python API is subject to change without advance notice. GI allows the bindings to be automatically updated, which is one of its advantages, but you may need to update your Python code. </para> </section> </chapter>