Using the GEDCOM parser library


Index

Overview

The GEDCOM parser library provides two interfaces.  At the one hand, it can be used as a callback-based parser (comparable to the SAX interface of XML); at the other hand, the parser can be used to convert the GEDCOM file into an object model (comparable to the DOM interface of XML).  It comes with:
Additionally, if you want to use the GEDCOM C object model, the following should be used (note that libgedcom.so is also needed in this case, because the object model uses the callback parser internally):
There is a separate script to help with library and compilation flags, see the development support.

Next to these, there is also a data directory in $PREFIX/share/gedcom-parse that contains some additional stuff, but which is not immediately important at first.  I'll leave the description of the data directory for later.

The very simplest call of the gedcom callback parser is simply the following piece of code (include of the gedcom.h header is assumed, as everywhere in this manual):
int result;
...
gedcom_init();
...
result = gedcom_parse_file("myfamily.ged");
Although this will not provide much information, one thing it does is parse the entire file and return the result.  The function returns 0 on success and 1 on failure.  No other information is available using this function only.

Alternatively, programs using the C object model should use the following (in this case, the inclusion of both gedcom.h and gom.h is required):
int result;
...
gedcom_init();
...
result = gom_parse_file("myfamily.ged");
The call to gom_parse_file will build the C object model, which is then a complete representation of the GEDCOM file.

No matter which of the interfaces you use, the call to gedcom_init() should be one of the first calls in your program.  The requirement is that it should come before the first call to iconv_open (part of the generic character set conversion feature) in the program, either by your program itself, or indirectly by the library calls it makes.  Practically, it should e.g. come before any calls to any GTK functions, because GTK uses iconv_open in its initialization.
 
For the same reason it is also advised to put the -lgedcom option on the linking of the program as the last option, so that its initialization code is run first.  In the case of using the C object model, the linking options should be: -lgedcom_gom -lgedcom

The function gedcom_init() also initializes locale handling by calling setlocale(LC_ALL, ""), in case the application would not do this (it doesn't hurt for the application to do the same).
 
The next sections will refine this piece of code to be able to have meaningful errors and the actual data that is in the file.

Error handling

The library can be used in several different circumstances, both terminal-based as GUI-based.  Therefore, it leaves the actual display of the error message up to the application.  For this, the application needs to register a callback before parsing the GEDCOM file, which will be called by the library on errors, warnings and messages.

A typical piece of code would be (gom_parse_file would be called in case the C object model is used):
void my_message_handler (Gedcom_msg_type type, char *msg)
{
  ...
}
...
gedcom_set_message_handler(my_message_handler);
...
result = gedcom_parse_file("myfamily.ged");
In the above piece of code, my_message_handler is the callback that will be called for errors (type=ERROR), warnings (type=WARNING) and messages (type=MESSAGE).  The callback must have the signature as in the example.  For errors, the msg passed to the callback will have the format:
Error on line <lineno>: <actual_message>
Note that the entire string will be properly internationalized, and encoded in UTF-8 (Why UTF-8?).  Also, no newline is appended, so that the application program can use it in any way it wants.  Warnings are similar, but use "Warning" instead of "Error".  Messages are plain text, without any prefix.

With this in place, the resulting code will already show errors and warnings produced by the parser, e.g. on the terminal if a simple printf is used in the message handler.

Data callback mechanism

The most important use of the parser is of course to get the data out of the GEDCOM file.  This section focuses on the callback mechanism (see here for the C object model).  In fact, the mechanism involves two levels.

The primary level is that each of the sections in a GEDCOM file is notified to the application code via a "start element" callback and an "end element" callback (much like in a SAX interface for XML), i.e. when a line containing a certain tag is parsed, the "start element" callback is called for that tag, and when all its subordinate lines with their tags have been processed, the "end element" callback is called for the original tag.  Since GEDCOM is hierarchical, this results in properly nested calls to appropriate "start element" and "end element" callbacks.

However, it would be typical for a genealogy program to support only a subset of the GEDCOM standard, certainly a program that is still under development.  Moreover, under GEDCOM it is allowed for an application to define its own tags, which will typically not  be supported by another application.  Still, in that case, data preservation is important; it would hardly be accepted that information that is not understood by a certain program is just removed.

Therefore, the second level of callbacks involves a "default callback".  An application needs to subscribe to callbacks for tags it does support, and need to provide a "default callback" which will be called for tags it doesn't support.  The application can then choose to just store the information that comes via the default callback in plain textual format.

After this introduction, let's see what the API looks like...

Start and end callbacks

Callbacks for records

As a simple example, we will get some information from the header of a GEDCOM file.  First, have a look at the following piece of code:
Gedcom_ctxt my_header_start_cb (Gedcom_rec rec,
                                int level,
                                Gedcom_val xref,
                                char *tag,
                                char *raw_value,
                                int parsed_tag,
                                Gedcom_val parsed_value)
{
  printf("The header starts\n");
  return (Gedcom_ctxt)1;
}

void my_header_end_cb (Gedcom_rec rec, Gedcom_ctxt self)
{
  printf("The header ends, context is %d\n", (int)self);   /* context will print as "1" */
}

...
gedcom_subscribe_to_record(REC_HEAD, my_header_start_cb, my_header_end_cb);
...
result = gedcom_parse_file("myfamily.ged");
Using the gedcom_subscribe_to_record function, the application requests to use the specified callbacks as start and end callback. The end callback is optional: you can pass NULL if you are not interested in the end callback.  The identifiers to use as first argument to the function (here REC_HEAD) are described in the interface details .  These are also passed as first argument in the callbacks (the Gedcom_rec argument).

From the name of the function it becomes clear that this function is specific to complete records.  For the separate elements in records there is another function, which we'll see shortly.  Again, the callbacks need to have the signatures as shown in the example.

The Gedcom_ctxt type that is used as a result of the start callback and as an argument to the end callback is vital for passing context necessary for the application.  This type is meant to be opaque; in fact, it's a void pointer, so you can pass anything via it.  The important thing to know is that the context that the application returns in the start callback will be passed in the end callback as an argument, and as we will see shortly, also to all the directly subordinate elements of the record.

The tag is the GEDCOM tag in string format, the parsed_tag is an integer, for which symbolic values are defined as TAG_HEAD, TAG_SOUR, TAG_DATA, ... and USERTAG for the application-specific tags.  These values are defined in the header gedcom-tags.h that is installed, and included via gedcom.h (so no need to include gedcom-tags.h yourself).

The example passes a simple integer as context, but an application could e.g. pass a struct (or an object in a C++ application) that will contain the information for the header.  In the end callback, the application could then e.g. do some finalizing operations on the struct to put it in its database.

(Note that the Gedcom_val type for the xref and parsed_value arguments was not discussed, see further for this)

Callbacks for elements

We will now retrieve the SOUR field (the name of the program that wrote the file) from the header:
Gedcom_ctxt my_header_source_start_cb(Gedcom_elt  elt,
                                      Gedcom_ctxt parent,
                                      int         level,
                                      char*       tag,
                                      char*       raw_value,
                                      int         parsed_tag,
                                      Gedcom_val  parsed_value)
{
  char *source = GEDCOM_STRING(parsed_value);
  printf("This file was written by %s\n", source);
  return parent;
}

void my_header_source_end_cb(Gedcom_elt  elt,
                             Gedcom_ctxt parent,
                             Gedcom_ctxt self,
                             Gedcom_val  parsed_value)
{
  printf("End of the source description\n");
}

...
gedcom_subscribe_to_element(ELT_HEAD_SOUR,
                            my_header_source_start_cb,
                            my_header_source_end_cb);
...
result = gedcom_parse_file("myfamily.ged");
The subscription mechanism for elements is similar, only the signatures of the callbacks differ.  The signature for the start callback shows that the context of the parent line (here e.g. the struct that describes the header) is passed to this start callback.  The callback itself returns here in this example the same context, but this can be its own context object of course.  The end callback is called with both the context of the parent and the context of itself, which in this example will be the same.  Again, the list of identifiers to use as a first argument for the subscription function are detailed in the interface details .  Again, these are passed as first argument in the callback (the Gedcom_elt argument).

If we look at the other arguments of the start callback, we see the level number (the initial number of the line in the GEDCOM file), the tag (e.g. "SOUR"), and then a raw value, a parsed tag and a parsed value.  The raw value is just the raw string that occurs as value on the line next to the tag (in UTF-8 encoding).  The parsed value is the meaningful value that is parsed from that raw string.  The parsed tag is described in the section for record callbacks above.

The Gedcom_val type is meant to be an opaque type.  The only thing that needs to be known about it is that it can contain specific data types, which have to be retrieved from it using pre-defined macros.  These data types are described in the interface details.

Some extra notes:

Default callbacks

As described above, an application doesn't always implement the entire GEDCOM spec, and application-specific tags may have been added by other applications.  To preserve this extra data anyway, a default callback can be registered by the application, as in the following example:
void my_default_cb (Gedcom_elt elt, Gedcom_ctxt parent, int level, char* tag, char* raw_value, int parsed_tag)
{
  ...
}

...
gedcom_set_default_callback(my_default_cb);
...
result = gedcom_parse_file("myfamily.ged");
This callback has a similar signature as the previous ones, but it doesn't contain a parsed value.  However, it does contain the parent context, that was returned by the application for the most specific containing tag that the application supported.

Suppose e.g. that this callback is called for some tags in the header that are specific to some other application, then our application could make sure that the parent context contains the struct or object that represents the header, and use the default callback here to add the level, tag and raw_value as plain text in a member of that struct or object, thus preserving the information.  The application can then write this out when the data is saved again in a GEDCOM file.  To make it more specific, consider the following example:
struct header {
  char* source;
  ...
  char* extra_text;
};

Gedcom_ctxt my_header_start_cb(Gedcom_rec rec, int level, Gedcom_val xref, char* tag, char *raw_value,
                               int parsed_tag, Gedcom_val parsed_value)
{
  struct header head = my_make_header_struct();
  return (Gedcom_ctxt)head;
}

void my_default_cb(Gedcom_elt elt, Gedcom_ctxt parent, int level, char* tag, char* raw_value, int parsed_tag)
{
  struct header head = (struct header)parent;
  my_header_add_to_extra_text(head, level, tag, raw_value);
}

gedcom_set_default_callback(my_default_cb);
gedcom_subscribe_to_record(REC_HEAD, my_header_start, NULL);
...
result = gedcom_parse_file(filename);
Note that the default callback will be called for any tag that isn't specifically subscribed upon by the application, and can thus be called in various contexts.  For simplicity, the example above doesn't take this into account (the parent could be of different types, depending on the context).

Note also that the default callback is not called when the parent context is NULL.  This is e.g. the case if none of the "upper" tags has been subscribed upon.

Support for writing GEDCOM files

The Gedcom parser library also contains functions to writing GEDCOM files.  Similar as for the parsing itself, there are two interfaces: an interface which is very basic, and requires you to call a function for each line in the GEDCOM file, and an interface which just dumps the Gedcom object model to a file in one shot (if you use the Gedcom object model).

Again, this section focuses on the basic interface, the Gedcom object model interface is described here.

Opening and closing files

The basic functions for opening and closing Gedcom files for writing are the following:
Gedcom_write_hndl gedcom_write_open (const char* filename);
int               gedcom_write_close (Gedcom_write_hndl hndl, int* total_conv_fails);
The function gedcom_write_open takes a parameter the name of the file to write, and returns a write handle, which needs to be used in subsequent functions.  It returns NULL in case of errors.

The function gedcom_write_close takes, next to the write handle, an integer pointer as parameter.  If you pass an actual pointer for this, the function will write in it the total number of conversion failures; you can pass NULL if you're not interested.  The function returns 0 in case of success, non-zero in case of failure.

Controlling some settings

Note that by default the file is written in ASCII encoding (and hence e.g. accented characters will cause a conversion failure).  You can change this by calling the following function before calling gedcom_write_open, i.e. it affects all files that are opened after it is being called:
int gedcom_write_set_encoding (const char* charset, Encoding width, Enc_bom bom);
The valid charset values are given in the first column in the file gedcom.enc in the data directory of gedcom-parse ($PREFIX/share/gedcom-parse).  The character sets UNICODE, ASCII and ANSEL are always supported (these are standard for GEDCOM), as well as ANSI (not standard), but there may be others.

The width parameter takes one of the following values:
The bom parameter determines whether a byte-order-mark should be written in the file in case of UNICODE encoding (usually preferred because it then clearly indicates the byte ordering).  It takes one of the following values:
For both these parameters you can pass 0 for non-UNICODE encodings, since that corresponds to the correct values (and is ignored anyway).  The function returns 0 in case of success, non-zero in case of error.  Note that you still need to pass the correct charset value for the HEAD.CHAR tag, otherwise you will get a warning, and the value will be forced to the correct value.

Further, it is possible to control the kind of line terminator that is used, via the following function (also to be used before gedcom_write_open):
int gedcom_write_set_line_terminator (Enc_line_end end);
The end parameter takes one of the following values:
By default, this is set to the appropriate line terminator on the current platform, so it only needs to be changed if there is some special reason for it.

Writing data

For actually writing the data, the principle is that every line in the GEDCOM file to write corresponds to a call to one of the following functions, except that CONT/CONC lines can be automatically taken care of.  Note that the resulting GEDCOM file should conform to the GEDCOM standard.  Several checks are built in already, and more will follow, to force this.  There is (currently) no compatibility mode for writing GEDCOM files.

In general, each of the following functions expect their input in UTF-8 encoding (see also here).  If this is not the case, errors will be returned.

Note that for examples of using these functions you can look at the sources for the Gedcom object model (e.g. the function write_header in gom/header.c).

Records

For writing lines corresponding to records (i.e. on level 0), the following function is available:
int gedcom_write_record_str (Gedcom_write_hndl hndl, Gedcom_rec rec, char* xrefstr, char* value);
The hndl parameter is the write handle that was returned by gedcom_write_open.  The rec parameter is one of the identifiers given in the first column in this table (except REC_USER: see below).  The xrefstr and val parameters are respectively the cross-reference key of the record (something like '@FAM01@'), and the value of the record line, which should be NULL for some record types, according to the same table.

Elements

For writing lines corresponding to elements (inside records, i.e. on a level bigger than 0), the following functions are available, depending on the data type:
int gedcom_write_element_str  (Gedcom_write_hndl hndl, Gedcom_elt elt, int parsed_tag,
                               int parent_rec_or_elt, char* value);
int gedcom_write_element_xref (Gedcom_write_hndl hndl, Gedcom_elt elt, int parsed_tag,
                               int parent_rec_or_elt, struct xref_value* value);
int gedcom_write_element_date (Gedcom_write_hndl hndl, Gedcom_elt elt, int parsed_tag,
                               int parent_rec_or_elt, struct date_value* value);
int gedcom_write_element_age  (Gedcom_write_hndl hndl, Gedcom_elt elt, int parsed_tag,
                               int parent_rec_or_elt, struct age_value* value);
These functions only differ in the type of the last argument, which is the value of the element.

The hndl parameter is again the write handle returned by gedcom_write_open.  The elt parameter is one of the identifiers given in the first column in this table (except ELT_USER: see below).  The parent_rec_or_elt is the corresponding rec or elt identifier of the logically enclosing statement: this will determine the level number written on the line, as the level number of the parent + 1.

Some of the identifiers can actually stand for different tags.  For this reason, the parsed_tag has to be passed for some of them.  This parsed tag is the same as was returned by the callback functions defined above, and is an identifier of the form TAG_name.  This parameter is needed whenever the second column in this table shows several possible tags (this is e.g. the case for ELT_SUB_FAM_EVT).

Note that for writing a date value, the given value should be valid, i.e. all its struct fields filled in properly and consistent.  This can be done by calling gedcom_normalize_date (see here).

User-defined tags

For user-defined tags (tags starting with an underscore), there are separate functions, again depending on the data type:
int gedcom_write_user_str  (Gedcom_write_hndl hndl, int level, char* tag, char* xrefstr,
                            char* value);
int gedcom_write_user_xref (Gedcom_write_hndl hndl, int level, char* tag, char* xrefstr,
                            struct xref_value* value);
In the case of user-defined tags, the level and tag string are passed verbatim (not controlled by the library).  This allows to write any extra data that doesn't use a standard tag, but is only allowed for tags starting with an underscore.

Other API functions

Although the above describes the basic interface of the gedcom parser, there are some other functions that allow to customize the behaviour of the library.  These will be explained in the current section.

Debugging

The library can generate various debugging output, not only from itself, but also the debugging output generated by the yacc parser.  By default, no debugging output is generated, but this can be customized using the following function:
void gedcom_set_debug_level (int level, FILE* trace_output)
The level can be one of the following values:
If the trace_output is NULL, debugging information will be written to stderr, otherwise the given file handle is used (which must be open).

Error treatment

One of the previous sections already described the callback to be registered to get error messages.  The library also allows to customize what happens on an error, using the following function:
void gedcom_set_error_handling (Gedcom_err_mech mechanism)
The mechanism can be one of:
This doesn't influence the generation of error or warning messages, only the behaviour of the parser and its return code.

Compatibility mode

Applications are not necessarily true to the GEDCOM spec (or use a different version than 5.5).  The intention is that the library is resilient to this, and goes in compatibility mode for files written by specific programs (detected via the HEAD.SOUR tag).  This compatibility mode can be enabled and disabled via the following function:
void gedcom_set_compat_handling (int enable_compat)
The argument can be:
Currently, there is a beginning for compatibility for ftree and Lifelines (3.0.2).

Converting character sets

All strings passed by the GEDCOM parser to the application are in UTF-8 encoding.  Typically, an application needs to convert this to something else to be able to display it.

The most common case is that the output character set is controlled by the locale mechanism (i.e. via the LANG, LC_ALL or LC_CTYPE environment variables), which also controls the gettext mechanism in the application.  

With gedcom-parse comes a library implementing help functions for UTF-8 encoding (see the documentation for this library).

Development support

Macro for configure.in

There is a macro available for use in configure.in for applications that are using autoconf to configure their sources.  The following macro checks whether the Gedcom parser library is available and whether its version is high enough:
AM_PATH_GEDCOM_PARSER([min_version,[action_if_found,[action_if_not_found,[modules]]]])
All the arguments are optional and default to 0.  E.g. to check for version 1.34.2, you would put in configure.in the following statement:
AM_PATH_GEDCOM_PARSER(1.34.2)
Note that version numbers now contains three parts (since version 0.20.0: this is also the first version in which this macro is available).

The macro also sets the variables GEDCOM_CFLAGS and GEDCOM_LIBS for use in Makefiles.  Typically, this would be done as follows in a Makefile.am:
bin_programs   = myprg
myprg_SOURCES  = myprg.c foo.c bar.c
INCLUDES       = @GEDCOM_CFLAGS@
LDADD          = @GEDCOM_LIBS@
If your program uses some extra modules, they can be passed as fourth argument in the macro, so that the CFLAGS and LIBS are correctly filled in.  Currently, the only available module is gom (the Gedcom object model).  For example:
AM_PATH_GEDCOM_PARSER(0.21.2, , ,gom)
To be able to use this macro in the sources of your application, you have three options:

There are three preprocessor symbols defined for version checks in the header (but their direct use is deprecated: please use the macro above):
The last one is equal to (GEDCOM_PARSE_VERSION_MAJOR * 1000) + GEDCOM_PARSE_VERSION_MINOR. As you see, this only checked the major and minor version, not the patch number, so this is obsolete.

Compilation and linking flags

Similar to other libraries, the GEDCOM parse library installs a script gedcom-config to help with compilation and linking flags for programs that don't use autoconf/automake.

To get compilation flags for your program, use (depending on whether you only use the callback parser, or also the GEDCOM object model):
gedcom-config --cflags
gedcom-config --cflags gom
Similarly, to get linking flags, use one of the following:
gedcom-config --libs
gedcom-config --libs gom
$Id$
$Name$