BetterGEDCOM – Introduction to Goal and Requirements

Purpose

We have written this document to introduce the Goal of the BetterGEDCOM project and some of the major Requirements that the project is considering. Because this is just an introduction, we will not update it frequently so if you want to see the latest situation on the Requirements, and find out whether they’ve been formally accepted or not, please go to the page “Better GEDCOM Requirements Catalog”. We make no promises that all these requirements will make it through into the finished standard, as we may find contradictions and lower priorities, while real life may cause us to discard or delay some of them. Nor do we claim that any of these ideas are unique to BetterGEDCOM.

This is an introduction to things described elsewhere – any discussions about the Goal or the Requirements should take place on the Discussion tab of the Better GEDCOM Requirements Catalog page. Please do not use the Discussion tab of this page. If, on the other hand, you don’t want to discuss the Goal or the Requirements but think I’ve introduced something wrong, please drop me, AdrianB38, a message. This page should be modified only by the moderators GThorud or AdrianB38.

Introduction

GEDCOM is an acronym for GEnealogical Data COMmunications, and it’s defined in a standard developed by the Family History Department of The Church of Jesus Christ of Latter-day Saints. Its purpose is to define a simple text file format to contain genealogical data. As a text file, it can be read – if not necessarily understood – by anyone, copied and then perhaps be physically transported by any standard means (e.g., placed on a disk or any other computer media, attached to an email, etc.), stored, and later imported into a computer program to be used and manipulated.

The last version of the GEDCOM Standard, Release 5.5, appeared in January 1995, so it’s not surprising that there are a number of ways in which people think that the standard can be improved on. See the page “Shortcomings of GEDCOM”.

Why me?

You might think that since you don’t send or receive GEDCOM files, this is nothing to do with you. Not so. The GEDCOM standard has been a major influence in the development of most of today’s genealogy software. The things that GEDCOM keeps information about (people and families, for instance), the list of what that information is (birth, marriage and death dates, occupations, for instance) – all these have been the basis for almost every genealogy program available today. This common view (we call it a Data Model) has made it easy to transfer data in various forms, not just GEDCOM files, between people, between programs, and into online services. But the single, common view has not significantly changed in years. If you want to transfer data about places, military units or the involvement of multiple people in one event, for instance – you simply can’t do it, except as free format text.
So let’s improve it.

The Goal of the BetterGEDCOM Project

BetterGEDCOM will be a file format for the exchange and long-term storage of genealogical data.
It will be more comprehensive than existing formats and so become the format of choice.

Note that BetterGEDCOM will be a file format, just like GEDCOM. And just like GEDCOM, you’ll be able to tell your genealogy program to produce a file, which you can send to someone. They can then import it into their genealogy program – or they can leave it, stored somewhere (indexed we hope!), waiting to be consulted. Of course, you might want to save the file yourself, just in case your current software or your service isn’t around any more – in which case you really need the new format to store everything of interest.

We want the new format to be more comprehensive than GEDCOM, so you’ll be able to draw up a proper history of those places that changed names, or where your relative’s regiment fought. And even if you never see a BetterGEDCOM file, because you “do it all on the Internet”, we want you to be able to track those places and regiments because the online guys will, we hope, use the new model of genealogy.

Some Major Requirements

Conversion

Perhaps the most important Requirement is this – that if you have some data in your existing genealogy program, you’ve got to be able to export all that data into a BetterGEDCOM file. It’s no use if BetterGEDCOM’s Data Model doesn’t have a slot for some of your most precious data. But this is also one of the most difficult Requirements to deal with, as we can look at what GEDCOM version 5.5 holds and at what some of the most popular genealogy programs hold, but there’s a limit to what one group of people can do.

What we will have to do is work with the developers of the genealogy programs, as they will be writing the modifications to their programs that will enable them to produce BetterGEDCOM files. If they find something that they can’t enter into a BetterGEDCOM file, then we need to have a conversation about changing the BetterGEDCOM standard. These problems, which can already arise when transferring files between existing programs, may occur because developers have made differing interpretations of the GEDCOM standard(s) or they have extended the standard in ways unique to their own software. With the co-operation of the developers, these interpretations and extensions need to be identified and assessed for going into BetterGEDCOM.

Multi-person events

Ever recorded the emigration of a family? Probably you wrote up an emigration event for one individual and then (if your program allowed it) copied that event to the other members. And what happened when you realised you’d misread the Ellis Island form and it wasn’t the “Carmania” that the family came across on, but the “Caronia”? Yes, wasn’t it a pain to alter each event for each person. You did do them all, didn’t you? Wouldn’t it be nice if there was just one event for the entire group? That’s what BetterGEDCOM wants to give you – not just for the emigration event but for all events: the ability to link all the affected people to one single event.

This idea of a multi-person event isn’t just about reducing the workload when altering text. Take a wedding for instance. In GEDCOM, the bride and groom are swiftly linked to the event. But what about witnesses? Different people do different things – some will create a user-defined event for “witnessed” against those witnesses, while others set up an Associated-Person linkage. Either way, the witness information is separate from the marriage event. BetterGEDCOM will allow you to record all the people against the one marriage event as you enter it, and say what their role was – e.g. bride, groom or witness, even the minister if you like. Similarly, god-parents could be attached directly to the baptism event.

Families

Multi-person events will also affect births. In GEDCOM, the birth event applies to one person only and the software needs to look at details of the family in order to work out who the parents are. However, there are (usually) three people involved in a birth or adoption – BetterGEDCOM will allow you to record all three people against the one event, and say what their role was – e.g. birth-child, father or mother. This has a neat consequence. If these biological and legal details are taken out of the family, then the family becomes a social construction, allowing informally linked children to be added in, without creating an adoption event that simply doesn’t correspond to any real event. So, using this idea, we can create a family where not all children are linked biologically or through any formal adoption. You could also document an uncertain child-father relationship without the additional creation of a family. You might even add in the Aunt who you always thought of as part of the family, or document cohabitants. Today, using the current GEDCOM structure to record parents means that, in the case of some transient relationships, we are automatically creating a family where one simply does not exist in any meaningful sense of the word. BetterGEDCOM will allow us to record that biology without creating a social family.

At the same time as making use of the revised structure for families, we can make the revisions necessary to record various other types of partnership, such as civil partnerships and co-habitation, defining them in terms of what they are, rather than what they are not.

Uncertainty and Disproved Facts

Life is fuzzy. Genealogy is even fuzzier. In the real world, almost nothing outside pure mathematics is 100% certain. But GEDCOM isn’t good at recording that lack of certainty across the board. Some things allow uncertainty – like “ABT 1066” – but there are still plenty of areas where GEDCOM cannot show uncertainty. BetterGEDCOM aims to record that uncertainty – for instance, you should be able to record a level of confidence (e.g. “possibly”) against a conclusion, instead of having to add a note that people don’t read because it’s printed further down the page. Uncertainty in BetterGEDCOM will apply to more than just dates – “Probably London, England” would be useful, as would marking up probable – but not certain – family relationships. By the way – there are two types of uncertainty here – just consider the phrase “Probably About 1066” compared to “Definitely About 1066”. We want BetterGEDCOM to be able to record both sorts.

In addition, it may prove useful to allow the recording of rejected conclusions to stop researchers revisiting old ground.

Places and Groups

Genealogy isn’t just about people – we need to understand what happens to places, for instance, otherwise we might imagine that someone moved town during their life, when it was simply the town that changed its name or merged with another. And what about pictures or maps relating to a place, or events that happened there, such as a fire or a battle? BetterGEDCOM should be able to keep this data in one structure for each place, a structure that can be referenced by, for example, events that happen in the place. The BetterGEDCOM format will make the transfer of such location-based information viable, from person to person, or within a user’s own applications.

BetterGEDCOM will also support the recording of historic data about groups of people, such as organisations, companies, regiments. You will be able to write some simple details of the history of the regiment, record that several people are members of that regiment, and the program will be able to link the regimental details to each of your people who are in the regiment. Again, BetterGEDCOM will facilitate the transfer of this information.

Evidence & Conclusion

Perhaps the biggest change that BetterGEDCOM proposes is the potential use of what we refer to as the Evidence & Conclusion Model.

In programs that are based on current GEDCOM, generally each person or family in real life have just one Individual or one Family record in the file. Occasionally, I find it useful to have two – for instance, I have George, son of Sarah, baptised in 1796, and another George, who marries Ann in 1820. Because his surname is so rare, I think these are probably the same person. If and when I demonstrate to myself that they are the same guy, I’ll merge these two individuals together. But I know from previous experience, that there’s a risk that I’ll get worried and want to know – which facts come from which George? Finding and restoring an old version of my data is hard work. Wouldn’t it be nice if I could keep all the bits separate, creating a new George that represents my latest conclusion, and keeping the original evidence, so that I could go back and review it later on? And if a new bit of evidence pops up later, which shows Sarah’s George was buried at the age of one, then I would like to be able to discard all the subsequent conclusions that depended on these two Georges being the same. At the moment, I would have to manually disentangle George’s story myself and run the risk of not removing all the bits that I should – or removing too much.

Keeping all this evidence and conclusions separately is what we refer to as the Evidence & Conclusion Model.

Let’s be quite clear – BetterGEDCOM is not saying that this is how you’ll have to work if you want to use a BetterGEDCOM compatible program. For a start, I’ve got 2,500 individuals in my file and I’m not going back to split out all the evidence and conclusions. I’m just going to keep everything on one Individual record per each of those 2,500 people. I might start to use these ideas for future people in my files – or I might not. In any case, having the ability in BetterGEDCOM to split evidence and conclusions doesn’t mean that the application software is going to be programmed to do it. So the Evidence & Conclusion Model may, or may not, be coming to a program that you use.

Research

A major area ripe for exploration is how people carry out their research – how they set their own objectives, track what they have done and record their conclusions. Some people will keep their research notes entirely separate from the GEDCOM file, others will just use the normal GEDCOM notes. Some programs allow recording of simple free-format notes, while others allow recording of detailed, structured information about objectives, task definition and progress, and the results and conclusions.

BetterGEDCOM should allow recording of this information in a way that allows transfer between programs with different capabilities. Some of this information might be only of interest to one researcher, but as many of us use more than one program, it would be useful if BetterGEDCOM facilitated transfer of that information between our own programs as well as between different people.

Many of us believe that BetterGEDCOM has the opportunity to set the common framework for recording the research process, in the same way that GEDCOM set the framework for recording the results of that research. And if that happens, then we may see many more people handling their research in a robust manner, which can only be of benefit to the study of genealogy.

International

The members of the BetterGEDCOM project are determined that BetterGEDCOM will not be optimised only for English speaking members of the Western world. We want to support the recording of information about real life in all cultures, countries, time periods and belief systems. We don’t want to be biased towards any one of these. Now, many people have attempted to expand genealogy software into these constituencies and we want to take those ideas forward. So, by using Unicode, BetterGEDCOM will be able to handle text and names expressed in most of the world's writing systems. By allowing people to record the structure of names more effectively than currently, BetterGEDCOM will help users and applications to sort on, for example, several surnames and parts of words in the names. For instance, if marked up properly, a program would know to sort “Honda” before “d’Hondt”. Then there are the different calendars that have been used throughout history, and are still used today, which we need to accommodate at least as well as GEDCOM does now. It is also important to allow the recording events from different cultures and belief systems.

Multimedia

It might be easy to concentrate on the BetterGEDCOM file and miss that this file will just be one component of people’s collection of information. Virtually everyone has a collection of files containing such things images, video, documents, links to web pages and so on, linked to records of events, people, etc. in their GEDCOM file. Users may describe these files with many techniques, such as tagging, and BetterGEDCOM will have to be aware of these things. But one major problem is that the current GEDCOM is not able to transfer this multimedia along with the GEDCOM file in a standardized way. BetterGEDCOM will provide what we refer to as a container structure, which will contain not just the BetterGEDCOM file itself, but also all that linked multimedia. This single structure can then be transferred or stored in a single file, which can be “unpacked” into the target file area without the need to manually alter every file path using the application program.

Sources and Citations

There are currently problems with transfer of source and citation data between many programs, due to application-specific extensions and varying support for the relevant structures in GEDCOM. These variations result in big differences between the amount and structure of source and citation information that various programs can record. Some provide the limited set of information in GEDCOM, while others support a larger set, for example, that described in the book “Evidence Explained” for US sources - and there are even different interpretations of that book. Further, a large number of source and citation schemes are in use in various archives and libraries, in various countries, and naturally, these schemes will change over time.

A BetterGEDCOM solution may therefore have to cater for an open-ended set of such schemes, including user defined ones, perhaps allowing transfer of field definitions and citation templates. As it is unlikely that all programs will be able to handle all these schemes, it is therefore important that BetterGEDCOM tries to establish some common data fields that allow transfer of crucial data between such advanced schemes and programs that only support a simpler, limited set of data fields.

Changeable

It would be nice to get everything perfect in the BetterGEDCOM Standard – we know we won’t, and the most obvious area that we’ll fall down in is the list of events and attributes, particularly those from beliefs that we aren’t familiar with. Many of us also have user-defined events or attributes to cover things that the writers of GEDCOM didn’t cover but which might be useful to many people now. What BetterGEDCOM wants to do is create some sort of registry that will allow swift proposal and addition of new event and attribute types. These won’t replace user-defined types, but they will mean that those of us who come along later, may be able to use the same types as other people.

And finally

There are numerous other requirements that the BetterGEDCOM project is gathering, such as the ability to format text with bold, italics and other forms of styling. These can be found in the BetterGEDCOM Requirements Catalogue.

Stuff...

The previous version of the Goals page is obsolete. Its content was carved apart to form the Goal and Requirements Catalogue.