BetterGEDCOM - Goal and Requirements


Co-Moderators: gthorud and AdrianB38

Please do not use the discussion tab of this page, use the Requirements Catalog.

The Goal of the BetterGEDCOM Project is:
BetterGEDCOM will be a file format for the exchange and long-term storage of genealogy and family history data.
It will be more comprehensive than existing formats and so become the format of choice.

The first step towards this goal is to identify and discuss user requirements, and technical requirements, that could be satisfied by the specification of the BetterGEDCOM file format. The work on requirements is organized around the following pages:

To avoid having discussions taking place all over the wiki, discussion of the goal and the requirements will be confined to pages or discussions pointed to in the Requirements Catalog. In the future, this page will be modified usually only by the moderator.

Archive

Comments

ACProctor 2011-12-06T13:13:23-08:00
Mission Statement
I see that the goal of BetterGEDCOM is currently listed as:

"BetterGEDCOM will be a file format for the exchange and long-term storage of genealogical data"

I apologise for digging this over again but should we distinguish between genealogical data and family-history data?

From the few threads I've participated in, I think most of us agree on the essential difference but would it recognised in the community & industry as a whole?

P.S. I added a link to BetterGEDCOM at the bottom of the wikipedia page on GEDCOM and quoted the current mission statement (under 'Alternatives'). Please let me know if this is not wanted and I'll removed it.

Tony
ACProctor 2011-12-06T13:37:36-08:00
Is it worth bringing this up briefly in wednesday's meeting? It doesn't sound too far removed from the subject matter.

Tony
GeneJ 2011-12-06T13:39:20-08:00
I'll put it on the agenda. --GJ
gthorud 2011-12-06T16:13:47-08:00
An earlier discussion on the same topic starting here, probably there are others.

https://bettergedcom.wikispaces.com/message/view/GOALS/32226276#32352006

I support the broadening of the scope.
WesleyJohnston 2011-12-07T03:28:59-08:00
I agree that the scope should include both genealogy and family history -- and also the other types of social links that ACProctor mentions, particularly local history.

FamilySearch Labs' "Community Trees" (histfam.familysearch.org) and Genealogy.net "Local Heritage Books" (http://www.ortsfamilienbuecher.de/lavelsloh/?lang=en) are already pushing GEDCOM way beyond the a single family in a single database, since these databases have many unconnected families and even unconnected individuals.

But we really need to keep an even broader scope in mind, which includes the standards for dates and places. Legacy Family Tree is already providing databases for standard places. And FamilySearch Labs has two such projects underway: (1) England Jurisdictions 1851 (maps.familysearch.org) and (2) "Standard Finder" (labs.familysearch.org/stdfinder/PlaceStandardLookup.jsp) for both dates and places. So these things are already happening, and if BetterGEDCOM is to ultimately be the independent standards authority for GEDCOM and all that relates to it, then these aspects must also be considered, or we will be foucsing on one aspect while the world has already begun tackling the big picture.
NeilJohnParker 2011-12-07T08:48:45-08:00
I concur with Wesley's comments re date and place standards, I believe Tom has already written a good document on date standards and we have already discussed place standards. As stated before I beieve place standrds should include 0 or more place name hierarchies which may be date range sensitive and have an type of the authority defining the place name heirarchy.
ACProctor 2011-12-07T09:19:29-08:00
I haven't seen Tom's document on dates. Can anyone point me to it? I've already posted some comments on the Date threads I found because there are standards for computer-readable dates.
GeneJ 2011-12-07T09:32:12-08:00
I'm sure I should post this in the Requirements Catalog ... but Louis Kessler posted an interesting article on GEDCOM dates this week.

http://www.beholdgenealogy.com/blog/?p=891
NeilJohnParker 2011-12-07T10:10:58-08:00
ttwetmore 2011-12-07T10:12:47-08:00
I haven't seen Tom's document on dates. Can anyone point me to it?

http://bartonstreet.com/deadends/DateFormats.pdf

Also accessible through the DeadEnds model page via the sidebar.
NeilJohnParker 2011-12-07T10:27:24-08:00
The fact that ACProctor couldn't find Tom's Data Defintions Paper on Dates suggests to me that in addition to the Data Model Section in our Table of Contents we need a section on Data Definitions along with the existing Glossary of Terms.
Dates and Places would be the first two entries, not to mention Person Names. I have never seen how we distinguish my sister-in-laws name Mary Ann which is actually her first and second name but it is not possible to know whether it is one or two name. Most geneaology programs have a seperate field for last name, they do not diferentiate between first and second and third etc names, rather they throw them all in a field called given names.
GeneJ 2011-12-08T04:57:43-08:00
@NeilJohnParker, re: Mary Ann

As you mention, different programs take different approaches to documenting name parts. While there may be additional features available, FTM-M uses a single entry field on its "add person" screen; RootsMagic uses a template [Given name(s); Surname; Prefix; Suffix; and Nickname]; Legacy uses a template [Given; Surname; Title Pre; Title Suf], TMG has a name template with 9 parts [Title; Prefix; GivenName; Surname; PreSurname; Surname; Suffix; OtherName; SortSurname SortGiven].

FamilySearch and Ancestry.com struggle with the presentation of names in their indexes, though I seem to recall FamilySearch has initiated some protocols. Smile. My ancestor's marriage record is indexed as "William Mr. Preston" and "Assenath Miss Butler." Fun thread some time back on a mailing list about Ancestry's "Alabama Marriage Collection, 1800-1869." Apparently during the indexing process, long s characters (as you might find in the word "Miss") became indexed as "p", so that volumes of brides became indexed as with the examples, "Mip Mary J Armstrong," "Mip Eleanor Andress," etc., etc.

See also Requirements Catalog, Person Names
(Data-PersonNames01: Sorting on multiple given names and surnames)
https://bettergedcom.wikispaces.com/Better+GEDCOM+Requirements+Catalog#Data-PersonNames01

Above Person Names entry in the Requirements Catalog references several wiki links, below:

Wiki page
http://bettergedcom.wikispaces.com/Person-Name+Elements

Names Discussion
http://bettergedcom.wikispaces.com/message/view/Person-Name+Elements/30777083

External Gramps page
http://gramps-project.org/wiki/index.php?title=GEPS_021:_Additional_Name_Fields
ACProctor 2011-12-09T02:25:22-08:00
Here's a good resource on this subject: http://www.w3.org/International/questions/qa-personal-names

On the newsgroups, I proposed a way of handling names in a culture-neutral way so that the concepts of forename/middle-name/surname are applied to the data by the genealogy product rather than being fundamentally represented in the data.

I did progress that work further if anyone is interested.

Tony
Andy3rd 2011-12-06T13:20:03-08:00
Tony,

I think adding the link on Wikipedia is a great idea. The more exposure we get the better as far as I'm concerned. I'm ashamed that I didn't think of doing that a long time ago.
;)
GeneJ 2011-12-06T13:20:22-08:00
Hi Tony,

During the first pass of discussions about the mission statement in the DTO, we developed the language below. The committee hasn't formulated a recommendation, but comments in this open forum are welcome.

[Name of Organization] brings together stakeholders from the genealogy and family history community for the purpose of developing open, international, technology standards and supporting documentation and services to benefit the community. All members ascribe to, and the organization operates in accordance with, the [name of organization] Code of Conduct*—we organize ourselves and operate without prejudice toward any existing or future genealogy software program or Internet service, ethnic group, culture or country.


The ad hoc committee did not consider the content of that code, just that a code would exist and be referenced in the statement of purpose.
GeneJ 2011-12-06T13:21:53-08:00
P.S. ... I love the link!
ACProctor 2011-12-06T13:33:31-08:00
I'm probably worrying unnecessarily. I went to copy the short goal and then wondered whether it sounded a bit restricted.

If GEDCOM is preoccupied with biological lineage then I believe BG should also be concerned with all types of social links between people, local history of places, and events in the lives of the people... IMHO of course ;-)

That sort of detail is probably better in a description of why BG was needed and what it offers that GEDCOM doesn't.

I link the idea of including both terms, as in the post above. Maybe that's all that's needed in the short goal.
GeneJ 2011-12-06T13:35:27-08:00
I'd sure prefer it say "genealogy and family history."
ACProctor 2011-12-07T04:47:14-08:00
Group Profile
After adding the BetterGEDCOM link to wikipedia, I started to wonder whether we should be raising our profile now or waiting until we're more structured.

I have an article being published in a UK Family History magazine in January so I asked my contact whether an article on our aims would be "sexy" enough to print.

I stressed it would not be endorsing any products as we're an independent group, and that it would be primarily focused on the history of GEDCOM, the problems with it, and the objectives of an updated version in scope and flexibility.

I think we would have to acknowledge that we're not the only initiative working in this field but the ones with the higher profile will be the ones that are listed to, consulted, and collaborated with.

Tony
Andy_Hatchett 2011-12-07T05:16:10-08:00
I tend to believe we should start raising our profile but maybe not be using a Brass Band to do so at this particular point. Thus, I see nothing wrong with the Wikipedia link or a small article and while I wouldn't be calling a news conference just yet, a presentation in the Exhibit Hall at RootsTech wouldn't be amiss either.

:)
ACProctor 2011-12-07T06:09:55-08:00
That's why I thought an informed article about the problem, rather than specifically the group itself, might be a good idea Andy. It would establish our credentials by our knowledge of the subject rather than using the "brass band" :-)

Tony
WesleyJohnston 2011-12-08T00:11:58-08:00
Sounds like a great idea
ACProctor 2011-12-08T03:09:08-08:00
Do we have any "collateral" that could be used, or reworked, for the press? I see lots of stuff over the wiki but wondered if anyone had already crafted a document describing the problems with existing file format(s) and the greater goals of the various initiatives underway.

For instance, some examples of the complex situations that get encountered, the goal of distinguishing evidence from conclusion, the ability to cite all sources accurately and have them appear in proper report form, the recording of events-in-time as opposed to simply names+BMD, the myriad types of source that genealogists want to cite, the relationship of the field to conventional historical research, etc.

There's a lot of material we could bring together to make a could case for what we're doing. I still think the stance of being part of a wider initiative sounds fairer than suggesting we're the only people working on this. It could also spur some unexpected collaborations.

I'm not really a word-smith - although I'm trying to move in that direction. I don't feel qualified to paint this picture the way the group wants, but I would certainly help with reviewing and suggestions if needed.

There's an interesting feature on the BBC here (http://www.bbc.co.uk/news/magazine-16000134) with a beautiful story of successful research. There's a suggestion that family-history could be part of the standard curriculum in schools. This is a great idea and would really boost children's interest in both subjects.
ACProctor 2011-12-09T02:52:54-08:00
Should I assume we don't have any such material? :-)

There are many people out there who just "don't get it" when discussing the need for a better data format. I've been depressed by some of the responses in related newsgroup discussions.

To have a set of stock articles that can be re-worked or re-jigged as appropriate is essential to get the message across in a clear and coherent way.

Some people see only the need for genealogical lineage. Some people see only the need for a computer exchange format (i.e. import/export) and claim GEDCOM already fits the bill.

My personal opinion is that the format that's needed will be far more important than most believe believe. It shouldn't ve viewed simply as an exchange format, and our mission statement correctly mentions long-term storage.

Also, I believe we shouldn't get bogged down in "holy wars" over lineage-based versus event-based versus evidence-based, etc. These can be accepted variations in the commercial products but our format - if it is to be universal - should be agnostic and represent all such aspects of the data.

...IMHO of course

Tony

P.S. Here's an article of event-based genealogy: http://hayesgreene.blogspot.com/2011/05/event-based-history-and-genealogy.html. I'm not saying I agree with it all but we need something that articulates how views.
GeneJ 2011-12-09T06:23:31-08:00
Hi Tony,

We sync, so more than ever, we need to be able to round trip the data. Some family/family object discussions on the wiki are below.

Somewhat related, there are a couple of current postings on Google+ about a new FTM report for "undocumented facts." It's again highlighting the inability to *source* a "relationship" in at least some of the programs that rely on the default family grouping--FTM, for example.


Requirements Catalog. See Data-Ind02 Biological ....
Discussion here: http://bettergedcom.wikispaces.com/message/view/Better+GEDCOM+Requirements+Catalog/34923376

Requirements Catalog. See Data-Fam01 ...
Discussion here: http://bettergedcom.wikispaces.com/message/view/Better+GEDCOM+Requirements+Catalog/34923732

Requirements Catalog. See Data-Fam02 Cohabitants
Discussion here:
http://bettergedcom.wikispaces.com/message/view/Better+GEDCOM+Requirements+Catalog/36660712

Discussion on the Gramps Data Model page
http://bettergedcom.wikispaces.com/message/view/GRAMPS+Data+Model/30087707

Discussion on OpenGen Data Model - "No Group or Family?"
http://bettergedcom.wikispaces.com/message/view/OpenGen+Data+Model/33871920

Discussion on Individual Data Elements discussion pag - Need for Family Entity?
http://bettergedcom.wikispaces.com/message/view/Individual+Data+Elements+Discussions/30448845

Aspects within discussion on Shortcomings of GEDCOM page - Families, step-families, biological families, etc.
http://bettergedcom.wikispaces.com/message/view/Shortcomings+of+GEDCOM/32069152

Page and related, "Family, Group and Relationship entities
http://bettergedcom.wikispaces.com/Family%2C+Group+and+Relationship+entities
GeneJ 2011-12-09T07:51:18-08:00
P.S. In Hayes article, TY for the link, he writes, "What [family based programs] are not so good for, however, are collecting information about people who you think might be related, but aren't sure of."

I see these as two issues. Do I want to record all the persons and all the known/proven relationships? In the alternative, am I trying add people and declare them "candidates" for some principle relationship role (father/mother; father-bio/mother bio, father-ado/mother-ado; father-step/mother-step, etc.)?
ttwetmore 2011-12-09T13:07:24-08:00
Tony wrote: Also, I believe we shouldn't get bogged down in "holy wars" over lineage-based versus event-based versus evidence-based, etc. These can be accepted variations in the commercial products but our format - if it is to be universal - should be agnostic and represent all such aspects of the data.

I fully agree with this. These are some of the thoughts behind the DeadEnds model. It integrates all of the above.
AdrianB38 2011-12-09T14:00:50-08:00
Tony - re materials for outside consumption. I did write a page "to introduce the Goal of the BetterGEDCOM project and some of the major Requirements that the project is considering" - see https://bettergedcom.wikispaces.com/Introduction+to+Goal+and+Requirements

It was designed for outside consumption and may cover some of what you're thinking of.
ACProctor 2011-12-09T14:11:55-08:00
Thanks Adrian. That's a great start.

I'd like to be able to get core material like that crafted into magazine-quality articles to entice the readership. You know - adding pictures, case histories, horror stories, pitfalls, and stuff like that.

You've covered a lot of the raw material but it's that gloss on the top that can make all the difference [No offence intended - I would probably have written it just the same as you did]

Tony
GeneJ 2011-12-10T13:34:38-08:00
Hi Tony,

Wiki link to the original press release and some of the Nov-Dec 2010 discussions about articles that mentioned BetterGEDCOM.

(1) Original press release
Press Releases, Nov 2010 (Wiki page)
http://bettergedcom.wikispaces.com/Press+Releases

(2) Links to many of the blogs articles written as a result of the original press release
"Why we ask for change," Nov 2010 (Discussion attached to Press Releases wiki page)
http://bettergedcom.wikispaces.com/message/view/Press+Releases/29952625

(3) "BetterGEDCOM in the news," Dec 2010
Discussion attached to Wiki home page; refers to Tampa Tribune and James Tanner articles.
http://bettergedcom.wikispaces.com/message/view/home/31656631

(4) "Comments elsewhere about GEDCOM" ("Buzz about BetterGEDCOM), Nov 2010
Discussion attached to home page
http://bettergedcom.wikispaces.com/message/view/home/29433017
ACProctor 2011-12-12T06:01:56-08:00
Thanks Gene. That's excellent!

Still no response yet from either magazine contact. I'll let you know when something comes back.

Tony