A major problem with the previous GEDCOM Standard was that it was never codified as a real standard. GEDCOM was (and technically still is) the product of one organization with its own specific goals. Unfortunately, when the organization determined it really wasn't within the scope of the organization's goals to be a universal genealogical standards developer, all development of the GEDCOM standard ceased. Now in fact the genealogical community is in danger of devolving into a world where little pockets of de facto, incompatible standards exist because no other effort has been made to make all these factions work together.

There are many nationally and internationally recognized organizations that could serve as bodies through which genealogical standards could be recognized. Here are a list of the most likely candidates:

IETF Internet Engineering Task Force - The only organization not part of the International Organization for Standardization (ISO), the IETF regulates the internet, and its standards, published as standards-track RFCs, define nearly everything in computer programming technology today.

NISO National Information Standards Organization - This organization is the standards body for things like libraries and the publishing industry. There are synergies with some of their standards, but this seems the least likely body with which to codify genealogical technology standards. They do codify some things like citation and reference standards for other disciplines.

INCITS InterNational Committee for Information Technology Standards - Involved more with computer hardware technology, they still do codify a number of software development reference standards.

AIIM Association for Information and Image Management - This body does some amazing work related to business process management, content management, document and records management, and Web 2.0 technologies. Initially they may not seem a good fit for a standard specifically to replace the previous GEDCOM standard, but in fact their recent work like the StratML Standard is very similar to GEDCOM. They might also be of enormous help in helping define a long-term road map for genealogical technology standards.

Comments

brianjd 2010-11-21T21:47:55-08:00
Standardizing
While, it is certainly desirable to have a standard, I question the need to get any outside body to give it's blessing. Except perhaps a Genealogical body.

I would certainly steer away from the ISO standard body. They do not have an open process whereby anyone can join in, and they would likely not want to provide the standard freely or even at a reasonable cost. Furthermore their behavior in granting as a standard the Microsoft XML document model clearly shows they can be influenced by rich and powerful interests. I would see no issues with the IETF, but would think getting national or international genealogy groups to buy into it would be a better approach.

Brian
greglamberson 2010-11-21T22:23:08-08:00
Well, firstly, I agree that such a process may not be absolutely necessary. I also agree that it is desirable.

Referring to ISO as one entity, however, is not at all reflective of reality. In fact, ISO is an umbrella group that, while it has its own processes, largely approves standards developed by affiliate groups with wildly different processes. I give specific examples of organizations through which standardization might be advisable, and even a cursory review of these organizations and their processes shows how they essentially have no bearing on each other regarding either processes or costs.

Personally I don't know anything more about the Microsoft XML DOM than I can google, but everything I read is about Microsoft's process through W3C (The World Wide Web Consortium), which has nothing to do with ISO in any way whatsoever. In any case, each standards body has to be evaluated independently, even when they fall under the ISO umbrella.

So what is the benefit to standardization? I see four major advantages:

1. Mediation during the standardization process will formally help resolve differences ;
2. Weight of formal recognition of the standard will drive adoption;
3.Codification of a standard with a standards body ensures that there is a framework by which to resolve ambiguities, conflicts in the use of the standard; and
4. Codification of a standard through a standards body ensures future development on the standard will not die.

Having done quite a bit of work on this, I suggest looking into the individual organizations and their processes. It's really impossible to even know what such a process might entail without examining these organizations separately and in some detail.

Oh, and yes, they generally do have ways you can be involved in the process without being some big organization or even paying a fee (cf. AIIM in particular).
dsblank 2010-11-22T03:41:34-08:00
I think that the MS Document standards fiasco that Brian mentions is this:

http://www.groklaw.net/staticpages/index.php?page=20051216153153504

This is a well-documented case where the normal standards process was completely bent to serve big business at the cost of a well-defined standard.

I agree with Brian that "getting national or international genealogy groups to buy into it would be a better approach."

In fact, I've suggested to BetterGEDCOM to get a Board of Directors sooner than later. Otherwise, you'll have standard without anyone using it. Vendors will not use BG unless they are made to by the users, and even then there will be constant battle to keep them from embracing and extending it.
greglamberson 2010-11-22T11:11:41-08:00
Doug,

I do recall some big mess several years ago, but what you're referring to at that link isn't apparent. In any case, none of that involves any of the actual organizations I have proposed that people look into.

Getting buy-in, developing a standard, and obtaining formal recognition of a standard are three entirely different things. This topic refers only to the third.

Your point regarding organization is one we discuss about every day in some fashion. It's not one that makes any sense to discuss openly, however, because there is no organizational plan that is ready to implement, there are sensitive issues that affect possible options. In the meantime it remains an advantage to remain a loosely-defined community-based effort.

Regarding buy-in, without something to buy into, it's silly to talk about other organizations' formal endorsement of our nonexistent finished work. However, if was do continue to move forward and emerge from this process with a working specification, you very well may be shocked at the widespread adoption that springs out of nowhere very quickly.

Back to the topic at hand. I understand that formal standardization is the answer to everyone's problems, but the potential upside in terms of a formal structure make it an option worth considering carefully instead of rejecting it outright.
brianjd 2010-11-23T20:54:23-08:00
I should clarify my previous comment. My two points are that any standard should be developed with input from within the genealogy community and care taken in choosing which organization has the best liklihood of being benficial.

Once the standard is hashed out, it can be submitted anywhere, but I think it would be wise to have some say in how it might in the future be made available, and how it would be maintained.

Very few ISO standards are available freely, and individuals and companies might not have any say in how it gets maintained. Who wants to pay $400 just to read a specification. If I'm going to write a piece of software to support a standard, I don't want to shell out $400 when I can come up with my own format for free.

Additionally, I was pointing out care should be taken in choosing which standards body to use. The Groklaw link is the fiasco I was talking about. It was an ISO standards process. It got really ugly. A total breakdown of the process.

Back to the standardization issue. Very few standards turn out being used s written. Only recently have most webbrowsers become compliant with the HTML standard. I've worked with various software standards for decades. I've yet to see two companies imlplement them the same way. As I like to say the great thing about standards is everyone has one. It also the worst thing. Whatever we develop will be chopped and "enhanced" by the software writers.

This sounds like a fun project. I'd be happy to help build it and write some modules that could be used to build/test compliance. Since, I've been loosely working on my own version [dusts off old charts] for some time.
greglamberson 2010-11-23T21:04:19-08:00
Brian,

Well, that all makes good sense.

That being said, what do you think of AIIM? I think it's a very intriguing possibility. Frankly, other than AIIM, I'm not all that encouraged by other options at the moment.
brianjd 2010-11-24T11:46:59-08:00
Well, first off, my disclaimer. I know very little about any standards body. I used to have a high opinion of ISO. I think any body is susceptible to the abuse that happened with the ISO.
I don't think our little standard will draw that kind of power play.

The AIIM, seems to be an older, if smaller standards body. It being born from the microfilm industry. It has a certain nostalgic appeal. THey are also pretty good at slef-advertising. Both the IIAM and ECM wikipedia articles are by IIAM power players.

Membership seems open to anyone, and the cost isn't too terribly steep. Company research isn't my forte, That means going to a library (some things require hardcopy research). I'm practically surgically attached to my laptop. ;')

I think there are many things we should consider, before choosing one.

Is the standards process open to anyone?

Does a person have to join the organization to participate? Is there always a cost in participating? Who gets a vote?

Does the standards body allow patent encumbrances in standards that would make it costly for developers?

Will the standard be freely implementable by anyone?

Will the standard be available for free to: the public, members, no one? If not, what will be the cost of obtaining a copy?

Are others prevented from disclosing it by some kind of license restriction?

How does the process work?

What is the cost?

That's just a few questions off the top of my head.
mstransky 2010-11-23T14:25:14-08:00
Stop re-inventing a source of label problems
Hello everyone, I am not a technical writer so I maybe hard to follow. I emailed Greg a thought where all the problems a make a standard are coming from.

I will take a crack at wording it.

The typical way gedcom tries to create a tag for every possible event out there. This makes a everyone suffer trying to create a Rosetta stone to export xml standards to other standards.

1. Everyone is creating various "tag named events" like immigration, baptism, birth, christening, and so on. This hinders religious beliefs, culture descriptions and so on.

I ask this all the above are events, all one has to do is seen if it happened at a place, time it was an EVENT. what people wish to call that or tag that has no importance at all on xml structure.

so look at it this way, your in a computer program or an online interface. Pull down a selection bar and pick Baptism, Birth, Immigration, Divorce, Murder, Land Purchase, Hair Color, DNA string, etc...

No Matter what what ever it was was done at a time or place, we need to stop creating date and place tags INSIDE an untold limitless description of tag names.

Lets reverse that and create a list of events names that can be select which fit in the standard time and place of event.

Solution:
whatever kind and type or label of an event should be on a standardized list which is selectable.
(choose list)
Brith
Death
DNA string
hair color
marriage
Census
draft
etc..etc...

That selected tag gone in the event node as a label.

So if people ever get around to converting db's they can import export the event with a standard tag name as text in the node corrosponsing to the event and person it lnks too.

what we will have is just a pure event and "type", then everyone can let go of make all those various node names including an addition date and place node in side hundreds of various tag names no one will even want to try and translate.

a simple event date place with text "type of event" can be translanted, filtered for and matched agaisnt one parse looking for a match.

Not creating hundreds of parses to match all the various demand types of events in an xml struture.

If someone else can say it in a clean technical wording please be my guest and do so if you grasped my opion on this.
mstransky 2010-11-24T06:57:41-08:00
Ok, I though I was the only one seeing that. I gave up 5 years ago on one board where people were almost fighting tooth and nail at each other over a religuos or cultural tag.

We can eliminate all of it by making a select choice of (labels) to put in the node as text.

If a app or parse does not understand "Baptizm" it can ask to you me Baptism" or "select other"

then either way a persons LINKED event will always display in Crono order by event date anyway

You can not die or be baptized before your born
that could be a good add on later in an app to catch.

My thought just create a PROPOSED list of Events that should be considered as an insert.

Later people may say what about such and such, then add them in an updated version.

@nd if others can see that with events, pull down(selectable records can be do the same why)

SOUR
TYPE selectable (Census, birth, land, burial, rollfilm, book, etc...)
CLSF classsification (1910 fed, 1900fed, purchse, certificate, deed, etc...
TITL NY, kings CO blah blah blah
(...more..)

this way no one company can hold the monopoly over the xml struture names, but focus othen on interpeting the text label for filter and match, or translating languages from one text to the other, NOT rewriting the structure of TAG NAMES

Simple
Make the xml universal on structure world wide
and the Labels selectable or most common/TEXT

Again thanks, ....wow 10 years waiting for progress, and the last 5 I had my head buried in my work.
cowe 2010-11-24T07:37:30-08:00
In my opinion event types are very poorly defined in Gedcom. Of course there are not enought event types, but could there ever be? Everyone is able to define their own event types using EVEN. It's however impossible for the receiver to know the meaning of it. If for example I create a Gedcom file with a user defined event of type "birth", no application would know to treat this as a BIRT event, and in the receiving application I would perhaps modify all these events, or worse, re-enter them, to make them appear as birth events and treated as birth events in reports.

So why does applications need to understand events? Only in a few cases, like in the example above. If the event type require no special treatment, the application doesn't need to understand it.

Another important issue is: What is the exact meaning of an event type? In my opinion, only the user knows. The application doesn't know exactly how the user has defined an event type(or interpreted the meaning of an existing event type). I would therefore argue that event types are a part of the dataset. It's not just properties of the application or data model. Consequently you can never translate the description of an event automatically, because you risk changing the meaning of that event type. If the meaning is ambigous (as it usually is) it should stay that way. It should be neither more nor less ambigous after import to another application.

My solution to those problems in GenXML was to introduce a limited number of event classes and make that a property of every event type. It's thus possible for the user to define an event type "drowned", and the application will still recognize it as a kind of death, because of the event class.

Regards,
Christoffer
ttwetmore 2010-11-24T07:55:28-08:00
Christoffer asks "So why does applications need to understand events?"

There is a very good answer. So it can understand the relationships between people. With Gedcom there is the family record and the various pointers (FAM, FAMS, HUSB, WIFE, CHIL), and this is the only way relationships can be represented. But with BG I don't think we want to be dependent on a family object to define all the relationships. So the relationships must be defined some how else. What better way would there be than to simply have the software look at the events and the roles the people play in them and just infer all that knowledge. That's how we humans do it. So we need to know a birth event is a birth event because that's how we know who is the mother, the father, and the child in a relationship. Some might argue, I suppose, that I've just made a compelling argument for the need for "relationship" entities, which I am not. There is a principle involved of not having redundant information in a model because of the problem of multiple places to update when changes occur. I think relationship objects are slippery, awkward things at best, and with persons and events already in the model, they are not needed. But I will listen to any arguments against that view is someone thinks there is something compelling. I'm not always right; just ask my wife.

By the way, I am not speaking abstractly here. I've written all the software to do these things, and know all the modeling ideas I describe are sound and work.

Tom Wetmore
mstransky 2010-11-24T07:56:38-08:00
Ok being on the same level.
"If the meaning is ambigous (as it usually is) it should stay that way"

You can ask many people where did you get that info

1. From a birth certificate
2. From a marriage record
3. story from my aunt

I dont see anything but a source document.
They are all events or EVNT or what ever system uses a tag.

They are all events, have a date, place, and title, and maybe a note. the only thing is like you say also the interpetation of it they can pick (choose from a editable list)

1. Birth , cert.
2. Marriage, liense, cert. etc., roll film,..
3. Story, Book, Letter, etc..

but all in all they are just documents, let the viewer come to their own conclusions to whom it is being referanced from.

do you think it is possible a majority of users mike like that idea to select a "TYPE" and "CLASS" over making an xml tag name for every possibilty? the standard would always change.

Keep it like I am saying, all you have to do is up date the choose list while the XML STRUCTURE did not have to change.

I think that is the big hurdle for anyone to getting to a standard xml structure world wide.
testuser42 2010-11-24T10:17:03-08:00
Christoffer, since you mention GenXML I checked it out and saw that you are the author of this specification. Great to have you here!

Maybe you would want to add GenXML to the list of data models with a little explanation?
Here are the links for others:
http://www.cosoft.org/genxml/index.html
http://www.cosoft.org/genxml/GenXML30.pdf

(I've just skimmed this PDF, but it looks like you've done a lot of very thorough work!)
testuser42 2010-11-24T10:24:28-08:00
Also, I think Christoffer's handling of events is a very good idea that hasn't been brought up here yet.

A limited number of event classes as a property of every event type would be very powerful. People could make up their own (regional) names for events, but the "main" events would still be compatible all over the world.
mstransky 2010-11-24T10:44:03-08:00
I like Christoffer's design and it it great and was. But there is two things

1. the design was for a purpose to cover data back then for it use.
2. Change and evolution of users demands.

prior to 2000 it was a majority of pedigree and some notes with those major events.

But now people are adding more then a birth and death cert. to the persons. Now there is census, drafts, and more coming.

People now what neat templates and generators spitting out timelines and graphs?

but it all comes back to birth, death, draft, census took place as a "Recorded Event" from a "Source of Info"

I am just seeing the evolutions of what people are wanting to do, yet we can not but favor to what is a major event or not, class them all fitting into the tag event. then use a "Recommended text" to describe it.
greglamberson 2010-11-24T12:51:32-08:00
Well, first off, I am starting to object to the idea of an Event. There are lots of things that aren't events that get shoved into this category because that's what we have had up to now.

I think this needs to be changed to FACT.

Christoffer, I knew your name from somewhere. I hope you'll add your info to the described data models and add a link to the sidebar. I haven't read the 3.0 spec and I can't right now..
Andy_Hatchett 2010-11-24T14:25:08-08:00
I'll agree that a new term may be needed, but I have a strong aversion to the term FACT- mainly due to genealogy programs using that term when what they usually mean is conclusion.
AdrianB38 2010-11-24T14:50:38-08:00
Greg - my own view is that we need Events and Characteristics (or Attributes as GEDCOM called them).

Characteristics (e.g. "physical description") have a mandatory value (e.g. "tall and thin") - Events don't have that.

In addition, Events can affect several people (or groups or locations) whereas Characteristics only make sense for one person (or group or location).

In the old GEDCOM we had events and attributes and it was quite difficult to split them apart - indeed, my software calls them both "Facts".

But now we have extended BG Events to be radically different from old GEDCOM Events by applying to multiple persons, the old idea of grouping the two as Facts has minimal advantage. I wouldn't want Fact to be used just for BG Events because of the possible confusion with the old GEDCOM fact that covered attributes as well.

Not sure about whether there are things getting shoved in as Events that aren't - where do I look for them?
mstransky 2010-11-24T15:31:48-08:00
Hmmm... fact or not, if someone has a photo, or draft card, or in the above "physical description") dont try to say it is not worthly enough to fit sort of.

Either of those above where done on a date, photo taken or doctors visit bote all examples below have date and time of occurance.

0 EVNT draft, WWI, NJ office
0 EVNT image, photo, family
0 EVNT medical, check up, FL central hospitol
0 EVNT census, 1910 fed, VA co 786
0 EVNT birth, certificate, FL central hospitol

Instead of saying which gathered materials above are worthy to be called facts or not, they are "still gathered facts in physical forms" in peoples scrap books.

YET on the other hand I can agree that it is hard to see all these as events.

Or better yet should we call EVENT as interpetaions of the FACTS

EVNT census, 1910 fed, VA co 786
to
INTR census, 1910 fed, VA co 786

Because some one can all ways come along and say that is Dillerd not Dellerd as the surname on the census writing.

That the document speak for itself, and the interpetation of it be just that. But understand all of them had a time and place they where documented and should not be cast aside as not worthy enough as BIRT or DEAT.
they are all equally important.
greglamberson 2010-11-24T16:05:47-08:00
For the sake of others interested in all these topics, could we please agree to carry these various discussions on in more appropriate places? This page has nothing to do with anything being discussed here, and some of these topics are very crucial. I'm very, very reluctant to continue discussing these things here, as others will never see this discussion.
mstransky 2010-11-24T04:37:02-08:00
example

Or you just label event
INDI@1234@
0 EVNT Christning
1 DATE some date
2 PLAC some plac
3 ROLL witness
4 SOUR @7777@
0 EVNT Christning
1 DATE some date
2 PLAC some plac
3 ROLL father
4 SOUR @7777@

Note above is note true GEDCOM !!!!

but instead of make a point in xml, I figured to show and example in GEDCOM style

Note Sour is the same, Event is label with a type of event, so if a person does not like Christening but preffers Baptism so be it.

2nd point, if the xml stand is done this way also, It is much easier to manipulate a universal tag EVENT with the type of in text, then it would ever be to change that actual NODE TAG name itself.

3rd for international languages, say from spanish or english, it would be easier to manipulate the text input of a NODE EVENT, than to stop function and open the XML and change the node name called from
<Christning>...
<date> <place> etc..</Christning>

To....
<Baptism>...
<date> <place> etc..</Baptism>

why not just

<event><type>Baptism</type><roll>Witness</roll>... <date> <place> etc..<sour>image 7777</sour></event>

So not matter what THE GENEOLIST does not have to re structure the XML itself, people can not debate about it no more. the labelling of the event is left up to and more flex able by the user.

lauguage support is better, if an importer from one platform to another would be much more flex able.
mstransky 2010-11-24T04:52:29-08:00
Everyone Tom, Greg and others

Don't take me like I am trying to push an agenda or steer you to think my way. It is very hard to express in text a concept or something I see might be causing a big problem.

But give me a chance to understand what I am pointing out and see if you can make it better for everyone. I guess I am trying to make the text label universal and getting away from locking in a tag name that traps the user or parser to comform to a TAG NODE NAME. Is it possiable for someone else to see what I am seeing and put it into better words?
testuser42 2010-11-24T06:26:12-08:00
Hi mstransky,

I think everybody agrees to your point. I haven't seen a proposal that said anything else about events anywhere here.
ACProctor 2012-05-15T13:17:13-07:00
Musings on Standardisation
Everyone agrees that a prerequisite for the development of a standard must be a requirements catalog(ue), i.e. a description of all the things it needs to support (such as cultural variations, edge cases, etc), and topics that haven't yet been resolved and so still need some constructive debate (such as at what point should support for sources+citations meet the "outside world").

However, there are some very high-level questions that precede even that. I created a list of about 10 such questions at Musings on Standardisation. Although I'm sure there are more than these 10, and my take on them won't be universal, it's still worth giving them a little thought. Without some basic agreement on what we're trying to solve, reaching agreement on how we should solve it will be even harder.

Tony