Comments

I read it as the British spelling of "program" with the definition of "a set of related measures, events, or activities with a particular long-term aim."

Tom

Synonyms for "programme" in this context could be "effort" or "project". My preference would be "project".

Tom

Oh dear - I thought I was safe with the word "programme", since I thought you all knew that even us old-fashioned Brits spell the word for "set of computer instructions" as "program".

My intention was that "programme" meant a series of work activities. I used that word since every bone in my body rebels at using the word "project" for this. It isn't a project - at least not yet - it doesn't have a defined beginning and end, and it certainly doesn't have project management! <grin>

However, I think I'm going to get more people asking "why / what programme?" than I am project management gurus complaining about the misuse of the word "project", so I'll swap to "project".

I guess I'm out of the loop then - I'm a Kiwi, so quite at home with the spelling of programme, but having lived in the US for 17 years (which is all but 4 of my Macintosh owning years), didn't realise that in English English, program and programme now co-existed with different meanings.

"Divided by a common language" is the phrase, I believe <grin>

Hope no-one starts mentioning "catalogue"!

Adrian - yes, apparently <g>

Catalog and catalogue, have the same meaning in standard American English.
As do program and programme.

They also, have numerous meanings in American English. The longer spellings are not in common use, but still considered correct.

That is really the root of the problem, using words that can be ambiguous, or words that have changed in meaning in recent history. Most Americans would not know for example that the British have separated out the alternate spellings of a word into to new words with separate and distinct meanings. ;')

My problem with modelling the evidence-conclusion process is that I don't believe we, as genealogists, truly yet understand what it is.

The GenTech model allegedly covers a lot of this - as I've seen in introductions to it - but frankly I don't understand how what's in their model relates to what I'd read the those introductions (which is where I got the idea of roll-back from).

Logically, the evidence-conclusion process starts with inputs, and finishes with outputs, where some of the outputs are
- the new (interim)-conclusions,
- a documented version of the logic that was gone through to reach those conclusions.

My issue is with the inputs, which _might_ include:
- previous interim conclusions;
- data from sources;
- assertions that we believe to be true (e.g. "Coverage of UK births in civil registration is complete enough for the possibility of a missing birth to be probabilistically insignificant")
- control data (e.g. dates of censuses)

Miss out any of those and the documentation of your evidence-conclusion process becomes an untruth. The process is therefore inherently risky and we certainly need to get our heads round the process before we can define what the inputs are in a clear and helpful manner. Chicken and egg.

The only sensible way of doing it that I've seen suggested so far is Tom's idea of generating conclusions from evidence because that avoids the human element. Yet that is so far from what the rest of us are doing....

I'll chime in here, even though I have no stake in subject. I don't see where it becomes the job of the standard to govern, control or dictate how evidence-conclusion works.

Taking out the human element is definitely wrong. "What is Toronto?", is all the evidence you should need to prove that. While computers can be very good at making decisions and doing lots of things. Drawing conclusions that depend on a deep understanding of Human language is NOT one of them. Yet.
While it is certainly true that giving the opportunity, humans will inevitably make bad evidence-conclusion relationships. People will do what they will to prove the conclusions they are pre-disposed to make. Our task is to give them a format that will allow them to make good ones.

If you build a standard to be a bigger-better-idiot-proof standard, God/nature will simply build a bigger-bigger-better idiot. Ad infinitum.

We must design for that 80% population, that require 20% of the work. That last 20% is a killer. For example to reach 90% it would require a 250% increase in work. To attain 95% would require approximately 350% as much work. Theoretically speaking. In actuality it is not given that any attempt to reach 100% on any given problem is actually achievable and that effort does not grow geometrically, rather than logarithmically. Sorry to introduce the math/s (word depending on your native language American/Everyone else).

Still wish there was a preview option.

@Adrian,

At least for me, the Genealogical Proof Standard (together with Mills' Research Process Map) explains the evidence-conclusion process.

I am not a fan of populating the database proper with evidence bits and person bits. I would just love it if those concepts could populate a research log. (So that conclusions could be entered to a database directly or also associated with a process documented in the research log.) --GJ

Brian - "I don't see where it becomes the job of the standard to govern, control or dictate how evidence-conclusion works"
I agree - that's why I don't want to do anything to inhibit better recording of it in BG, but can't see yet what we can do in BG.
I am interested in what machines can do with the data to turn it into information / conclusions - I just don't see it being more than marginally useful in the forseeable future for all the reasons you suggest.

Gene - I'm perfectly happy with the GPS and ESM's Research Process Map being an explanation of the evidence-conclusion _process_.

This Requirement is meant to be about recording the intermediate bits of data in the Data Model. Slightly too late I remember I put it into the Glossary as "Evidence and Conclusion Model" separate from "Evidence and Conclusion Process"

Your comment about a research log sounds very interesting - it isn't that far from the stuff in the "Evidence and Conclusion Model". Much as I like it, I hesitate before adding a Research Log as a requirement, else it'll be a never-ending list and it may be better to keep this to applications more specific to to-do handling, etc. But I will think about it...

Adrian,

I think we could easily add a minimal ability to have a research log without burdening the standard.

All we really need is a place to put research notes. A single tag should do it. We could call it Research Note, or Research Task, or it could be a particular type of a Task tag.

Entity: Task
Type: Research
Fields: Description, researcher, date started, date completed, isComplete, status [proved/disproved/inconclusive],owning record...


I created a Task entity last night, thinking along these lines. I made is very desirable, IIRC.

It could be used for much more than just a research log. A good way to track all the work done. Handily in one place, for easy retrieval, and to pass along to the next researcher. Good to have by your side in a fight. ;')

:)

What recent events are you speaking of?

I'm not sure if the BetterGEDCOM project has the resources to create a validation application. That's why I didn't put it in. Not having one is certainly a risk - just look at the @@@@ created in the name of GEDCOM.

Various people thought the only practical way was to create some "reference files" in the new format and so approach it from the other end, i.e. can your application read a complex, valid BG file?

I have offered, repeatedly, to write the application for the group and release it under a OSS license. GPL version 3 would be my choice for work I'm giving to a group. GPL v2 would be my choice for my own work. I think GPL v3 would be best for the group. For legal reasons.

When Better GEDCOM reaches the point of having formats to test, I will be tracking that progress with software that reads data in the BG format. Like Brian's offer, this software could be used as the basis of a validation program. My only proviso is that since my retirement, I only develop in Objective-C using the Foundation and Cocoa frameworks for the Mac OS X. Retirement == freedom forever from the Microsoft hegemony.

Brian - that would sound a good idea. Can I suggest you add it to the Requirements Catalog with reasoning and way forward as you have written above? Make it should or could depending on how much obligation you want to put on yourself. It would certainly offer a better means of testing compliance.

Then one of us can alter the scope to match.

Adrian

Well, if I deleted the text, it was not intended.

In 90% of the cases, all dates in a file will be using the same calendar, so if that calendar is identified in the file, I don’t understand the need to see the date encoded in text in order to identify the calendar. But I have proposed to have a sort date accompanied by a string, so what you want is possible even with a numerically encoded date.

The issue with errors is something that is handled by applications.

“Data Model could include a facility for a default calendar, etc, or this could be left to data entry or conversion in applications” I have no idea about what the last halve of this statement means.

The order of day, month and year is not dependent on the calendar, there is a huge number of ways to write a date in the Gregorian calendar.

Myrt has given some great lectures on dating issues.

I have only a handful of date entries (mostly my Norwegian baptisms) that are not either modern or limited location O/S dates.

To complicate the issue of my experience, I use a program that has fields for both "date" and "sort date."

I am not familiar with all the genealogical programs on the market, but do recall GENBOX a calendar like step when I entered an O/S date to that program.

Would sure like to see comment on this discussion from wide range of users. --GJ

Geir - a "sort date accompanied by a string, so what you want is possible even with a numerically encoded date". That sounds a useful way forward.

"order of day, month and year is not dependent on the calendar, there is a huge number of ways to write a date in the Gregorian calendar" - good point. The description does include "This [calendar] definition should include the ordering of the date items within the date" but perhaps I will tweak it to say "This [calendar] definition should be accompanied by a definition of the ordering of the date items within the date" to make it clear that these are 2 separate concepts.

"Data Model could include a facility for a default calendar, etc, or this could be left to data entry or conversion in applications" Yes.... It's not wholly clear on reflection.

Let me see if I can get it better - I'm trying to allay fears that people may have about suddenly going into their family history files and needing to add a calendar "code" to all their dates. This can be done by explicitly adding a default into the BetterGEDCOM file itself or by getting the application to update every date automatically. This latter would be done when converting from GEDCOM-compatible to BG-compatible data. I'm probably getting too deep into solutions here so I think I'll put something like
"To be decided: whether Data Model includes facility for a default calendar, etc, or whether every date must be marked up by calendar code, etc. Intelligent application design should reduce the workload for the user in either case."

PS - Geir - a minute or two ago, you were editing at the same time as me. Wikispaces told me what it thought your changes were and I think it has managed to include both our edits.

When I went back in to change a spelling mistake it offered me an unsaved draft to work from. I've no idea what was in that draft but wonder if it might have contained only my changes - or only yours - so such oddities might explain why Wikispaces' idea of your amendments might not match your memory. At any rate, I prefer to work on this before the US wakes up to reduce the risk of double changes.

Gene - we may need to brush up on Myrt's lectures! So far as I know, all Europe is now on the Gregorian calendar (willing to be proven wrong there...) So it seems logical to have one calendar code for that.

FYI - Before 1752, the calendar for England used the Julian calendar AND the year started on Lady Day, 25 March. I have an idea that Scotland used the same Julian calendar but started the year on 1 January. Indeed, there are English parish registers that sometimes started the year on 1 January! So how many codes do we need? Gulp!

And .. if we default to the Gregorian, then would you only need a code if you are entering the old style date (vs the indirect Gregorian equivalent).

Isn't that rather like existing GEDCOM, only expanded? (For some reason, I thought existing GEDCOM recognized the OS command following an old style date entry.)

P.S. Myrt's lecture is just packed with info. As to just that "1752 change" though, see Wiki for Gregorian Calendar. In particular, "Pope Gregory XIII, after whom the calendar was named, by a decree signed on 24 February 1582, a papal bull known by its opening words Inter gravissimas.[4] The reformed calendar was adopted later that year by a handful of countries, with other countries adopting it over the following centuries." [With emphasis on the word CENTURIES.]

On the default date and date format subject.

There is no need for worry on this. Everyone has already chosen or let alone the pre-chosen date format.

Possibly the application the user is using has it set. If not I guarantee the Operating System does.
The obvious solution is to simply presume the OS default unless there is a explicit override. Naturally, this will cause no end of problems, because there will be plenty of cases where the dates are in a format other than the expected. But there is no solution we could come up with that won't cause issues. we just have to accept that.

Brian, certainly no problem with there being a default on the PC, though while the O/S may well have set a calendar and date format, we need it out of there and into the file when we transmit it. No particular problem with that, of course.

And it might be the wrong format, of course. Ubuntu may well be wonderful but I somehow doubt it carries a calendar to give dates like 1 Jan 1666 OS / 1667 NS <grin>

(Gene - I enter these OS/NS dates into Family Historian quite happily so GEDCOM may well take them by default)

Brian,

There are problems with editing the page, the only solution I see is to save frequently - and cancel your edits since last save if you get a message that someone else is editing. Saved drafts - I am not sure - but tend to avoid them.

Isn't it possible to add a Country code in addition to the calendar value, or are there several dates for transition to Gregorian within a country - well Scotland - but maybe there is some ISO standard with a code for Scotland.

I agree that Date values must have an associated Calendar, although there should be a default calendar for, say, Gregorian - or whatever your most common Date value is relative to.

I have worked in the area of "globalisation" for many years and I feel strongly that BG should not get hung up on date-ordering issues, or use any day-names or month-names in date values - both of which would create a locale dependency. ISO 8601 was created as a Date standard for exchange and storage of date values, and the numeric format in particular (yyyy-mm-dd) is ideally suited to locale-neutral data values such as in XML.

All issues of date-entry and date-formatting are for the software loading or generating a BG file. The file itself must remain independent of those processes. Date-parsing should not be a consideration for BG as a locale-neutral and culture-neutral textual data format.

I am seeing again an issue that I have seen in a number of discussions. There are two very important perpectives that I believe have to be kept in awareness for BetterGEDCOM.

The first perspective is that of the researcher using BetterGEDCOM to create from sources a database that reflects the evidence and conclusions the researcher has used. This perspective includes both the storage of that database and the sharing of it, either with others or with merging with other databases of one's own. The vast majority of discussion posts come from this perspective.

The second perspective is of a repository that seeks to make available their records in BetterGEDCOM format. This is the perspective that I believe is being left out of most of the discussions.

And in this discussion of Date02, the second perspective is very important to retain in our awareness as we develop a standard. I think it is retained in the "Why?" statement of the requirement: "Dates may occur in source documents in all sorts of calendar representations. It is desirable that the codified representation of that should differ as little as possible from the written characters in the source, to reduce the scope for error in input or output."

If I am a curator of a repository, seeking to offer my repository information online in BetterGEDCOM format, then I do not want to be forced to convert dates in the source material into standard modern Gregorian dates (or any other dates either). If a record says that the event happened on some ecclesiastically named day of a year which is also likely to be an ecclesiastical year (e.g. not beginning on January 1), then I want to record the date as it is written.

I see an issue here very similar to place hierarchies: there is a need for standardized accepted databases of place hierarchies over time, and there is a need for standardized accepted conversion software to convert dates from all the different standards used into modern standard Gregorian dates, so that comparison of dates can be done on an apples to apples basis.

In both cases, this means some standards organization being responsible for the creation of those standards.

BetterGEDCOM must, in my view, support robust apples to apples comparison (which inherently includes ordering) of dates. This does mean parsing of dates and not merely carrying them as text strings. Certainly the early versions of BetterGEDCOM can cut corners on this, for both dates and places. But we should not lose the long-term vision of having these standard place and date aspects robustly supported, which does mean that ultimately someone has to do a whole lot of work to create and maintain those standards for places and dates.

I'm not sure I catch your drift Wesley.

Thinking just of Gregorian dates for a second, there might be 3 versions of a date in a typical case: the image or original document in which there is a written date, the transcribed date string, and the date value which is the interpreted version of that date. It is the latter of these - the machine-readable, searchable, sortable date value - that I am talking of. I appreciate that the other versions will need support but the date value must be unambiguous (assuming you can decipher the written version) and ISO 8601 is designed specifically for that purpose.

As you rightly point out, this is similar to place hierarchies in that you may have the original, a transcribed version (incl. any spelling errors and informality) , and the normalised machine-readable, searchable, sortable version.

Any scheme that has to depend on some magic date parser is doomed to failure because no such beast exists. You might be able to get one that works for the US, or for the UK, but a globalised bullet-proof on e is just not possible and I wouldn't trust any software that claim it can do it because I know there is no well-defined grammar and in the worst case the date is ambiguous.

re ACProctor

The different dates must be retained, and I see that your post was in regard to the unambiguous interpreted date. And it is good that you have raised the ISO 8601 standard.

I do think we need to have both a long-term view of bringing about just such a "magic" date parser, which is never going to be bullet-proof but probably could handle the vast majority of what it sees.

The fact that the reality of today is not close to that should not take it off the table for the long-term as something BetterGEDCOM as an organization should foster, while at the same time acknowledging that we have to build BetterGEDCOM version 1 within the constraints of where the technology is today.

I think it's reasonable that BG should mandate saying what the calendar is. (I've altered "should" to "must" to match).

I'm not sure I quite approve of the removal of the comments in the "Way Forward" section.

The comment about "It would probably seem sensible to define a default calendar and date ordering for each file" was intended to stop people moaning that they didn't want to mark every date in their file with a calendar. Yes it's a solution-comment not a requirement-comment but I felt it useful.

Also removed is the sentence "An alternative method would be to encode each date into the same representation - e.g. number of days since some agreed event." I put that in because it was mentioned as one possibility for "normalising" dates to allow translation. It's not a solution I liked because it's hard to deal with dates like "About January 1866" (as distinct from "About 1 January 1866") so I did reject it but felt it useful to record the rejection.

Comments anyone on whether the so-called clarifications add anything?

Geir - if I understand the outputs from this Wiki, you've altered "A BetterGEDCOM file must define the calendar" to "A BetterGEDCOM file should define the calendar", while leaving the importance as mandatory.

Perhaps I should have put into the template:
"must" goes with "mandatory" and
"should" goes with "very desirable".
The meanings of the words and phrases are intended to match, because "should" has an element of "should but it might not".

Do you agree with "mandatory" as the importance of the calendar?

Hm, my memory may not be the best, but I can't remember to have changed it.

But since you mention it, what is the intended use of Importance?

Also, I think that Way forward could include technical solution things - one may need to discuss that. The important thing is to describe what we want so people can understand it.

I understand the difference between should and shall. ISO editor guidelines has taken should out of the vocabulary. But it is difficult to focus on this all the time.

I agree with mandatory, and that the header should give the default.

I am not sure about the implications of "It is desirable that the codified representation of that should differ as little as possible from the written characters in the source, to reduce the scope for error in input or output." - does this mean that we should have text representation and not numeric - maybe not.

You are right about the Calender being dependent on country.

I think we need to discuss before we edit other people's text.

Adrian,

Sorry about removing that sentence, I was just trying to keep it in sync with your other comments. But as you say it's a solution comment. I thought it was there because you wanted to explain the Desirable status. Since it's naturally a software implementation issue which won't be dictated by by us. One thing I would like to suggest is that a calendar can also be "text only". That being the user entered some funky date that the program can't decipher. Setting Gregorian as the default calendar is probably going to correct 66% of the time.

Plus I thought this page was for the group. I also made a few minor edits fixing grammar in places. I am probably remember wrong, but I thought I saw a posting asking for help with this page. I keep pretty busy and get a ton of communications, so I sometimes mix up messages.

Geir - the history seemed to say to me that it was your id that made the change but I have my doubts that this Wiki always tells the truth if 2 people are updating at the same time. I note that you "agree with mandatory" so I will set it to "must" in the description.

Importance is there to help prioritise items.

Re "It is desirable that the codified representation of that should differ as little as possible from the written" - my thought there is that if the user enters "3 April 2001" (say) and then looks at the GEDCOM, it seems more useful if they recognise the value as something like they entered. It's just something to think about when evaluating how the date should be stored on the file. Personally, I don't like the idea of the numeric date partly because it gives no clue about the calendar in use - e.g. is 2 = February or Brumaire? Though if we explicitly mark each date with a calendar code, maybe it doesn't matter.

Brian - please don't feel you shouldn't update the page. In other places we have gone through loads of discussions but no-one ever updated the main page - hence when I was trawling for the first cut of requirements I nearly lost the will to live after a while!

I've put back a bit about possibly having a default calendar (modified to take account of your comment about it possibly being an application issue)

Plus I've highlighted the assumption that we are going to work with calendar dates and not days elapsed since something.

Probably, possibly, likely, maybe, unlikely, et al are all conclusions. These values are mostly relevant to an evidence-conclusion model.

Contrast "between 12/1899 and 11/1900"
with "probably 1900" and "about 1900".

The first one is a statement based on an age listed at the time of death on a record.

The second is a conclusion based on that fact.

The third, is either a cop out on the part of the data entry person, a shorthand way of saying "between date x and y", or a conclusion and is hence a more neutral term. Plus "about" is the de facto standard way of saying maybe and possibly.

In the end "probably", et al is a conclusion, if not taken from a fact (ex: a gravestone giving date range 1846-1900).

We're mixing apples and oranges.

The thread I started (URL above) had nothing to do with the evidence-conclusion model that Tom and Louis came to discuss.

Rather, the thread proposed BetterGEDCOM advance the more absolute style of current GEDCOM records into something far more relevant.

Solid conclusions, such as those resulting from a well reasoned proof argument, are often crippled when users are confined to the more absolute style of GEDCOM.

I'm guessing more dialog is necessary to get us all on the same page with this topic in the lead up to finalizing the writing of this "requirement."

Gene,

I re-read your original post, again. I guess I spaced it out a bit. First off, I use Gramps as my DB of choice. It allows me to put in "about","before","after" and give date ranges for events.

I see no reason why BG should not support that. It's only logical, and thus we should make that clear whether BG will support that too.

However, you are mixing metaphors so to speak.
There's a big difference between
"born between 12/1/1899 and 3/31/1900"
and "born between 12/1/1899 and 3/31/1900, probably in Manchester, England".

The first is merely recording a date range based on evidence, and the second adds in a conclusion with no supporting evidence except perhaps that the alleged parents lived there in that time range.

I don't see a problem with adding in qualifiers like this for dates, names, places, etc. But I think we need to be careful and specific in applying that.

Solid conclusions are still conclusions. If we have apples and oranges, it's because you asked about apples and oranges, hoping to get the oranges renamed to apples. You mixed questions about using facts to limit possibilities (born between, died between), with using known facts to draw conclusions (born probably at).

One of your examples could have gone either way. A child being baptized by the parents may or may not indicate legitimacy or marriage, or may incorrectly indicate one state over the other.

I wrote the original text Gene, and for once I do remember what I meant and no, the specific requirement is NOT intended to cover concepts like "Probably 1812", i.e. it is not intended to address the original point of that thread.

No reason there shouldn't be another requirement about that but it should be a separate requirement because "About 1812" and "Probably 1812" are different concepts - you can see it better if you tweak the phrases to be "I know it was about 1812" and "Probably 1 September 1812 but I can't be certain"

One's fuzzy on the date, the other's fuzzy on the certainty.

Yes, it would be useful if we could agree these terms but I really don't see it as BG's job to do that. I'll let someone else take on the massed ranks of American Family Historians - sorry, Genealogists! (What hope have we got if we can't even agree what to call ourselves! <grin>)

Tertiary is logical but we also then get into debate of whether a photo of a photo of an original is tertiary, secondary or primary. And of course, the answer may not be the same for a .JPG

And _personally_ I'm still sticking with my view that the UK thinks that Primary has to be contemporary. But, as you say, these are just short-hand for a 1st cut stab at how reliable the source is.

In my mind, and I'm not alone in this, there is a very distinct difference between being a Genealogist and being a Family Historian.

Andy: There is a very distinct difference in my mind also, but I suspect it may be the exact opposite of the one in your mind! More of the UK and the USA being divided by our common language. Like I say, let's not go there....

I think all of us here are interested in careful research into all aspects of our relatives' lives, no matter what we call it!

Perhaps this should be titled, "Source Form, Information Class and Evidence Type."

SOURCE FORM: Original or Derivative (with derivatives able to be further categorized as "abstracts, compilations, databases, extracts, transcripts, translations, and authored works such as genealogies and histories"). [Mills, _Research process Map_, 2006, and _Evidence Explained_, 2007, p. 24-25.]

INFORMATION CLASS: Primary information or Secondary information. [Mills, _Research process Map_, 2006, and _Evidence Explained_, 2007, p. 24-25.]

EVIDENCE TYPE: Direct, Indirect or Negative. [Mills, _Research process Map_, 2006, and _Evidence Explained_, 2007, p. 24-25.]

Is this a way forward?

I happened upon this APG-L message again, so I thought I'd post it.

http://archiver.rootsweb.ancestry.com/th/read/apg/2004-08/1092707376

Mills, "Citation and facts," 16 Aug 2004, APG-L, in part, as below:

If only we could just get those genealogy programs to do away with that ephemeral, ambiguous "surety" concept and replace it with check-boxes for the concrete descriptions used by modern evidence analysis -- i.e.,
Source: Original or derivative
Information: Primary or secondary
Evidence: Direct or indirect

What about using the following to weigh evidence:

Ask the following questions:

Did the information about the conclusion come from
- 1st hand knowledge/personal witness of an individual?
- 2nd hand knowledge of an individual?
- Another source document/record?

How does the information support the conclusion?
- Direct evidence?
- Indirect evidence?

Was the information
- Recorded at or near the time of the event?
- Recorded sometime after the event?

Is the information
- Clear and easy to read?
- Difficult to read?

Did the person/organization that recorded the information have any reason to alter the information?

Did the person/organization the provided the information have any reason to alter the information to hide or change what really happened?

Hiya:

Reference?

Gene, you said
If only we could just get those genealogy programs to do away with that ephemeral, ambiguous "surety" concept and replace it with check-boxes for the concrete descriptions used by modern evidence analysis -- i.e.,
Source: Original or derivative
Information: Primary or secondary
Evidence: Direct or indirect

Well, we can and should make these flags possible in BG. So then new software would be enticed to offer these checkboxes.

Mike's questions are more detailed, but the "flags" are similar.
I believe some of these flags would be best attached to "Source" records, others to "Evidence" records (or Extractions in Mike's model), and others to "Conclusions".

Can checkboxes or flags do everything that's necessary for determining the quality of Sources and Conclusions? Or do we need more flexibilty?
Would we still need a generic "surety"?

(1) Surety. See the requirement for QUAY and the related discussion. As I recall, there is a particular relevant linked discussion bout QUAY. Quay is relevant to many users at the application level-to eliminate QUAY would remove a feature meaningful to users. Our focus turned to clarifying the standard and re-assert the specific scale (0-3)
If we need to, it's better to continue a discussion about QUAY under the Requirement Catalog entry for QUAY.

(2) Mike's questions. There are probably two issues here. First, should we write specific and detailed questions such as in Mike's fine example, into a standard? Said another way, maybe vendor A sees a need for one set of helpful questions, but vendor B would like those questions phrased or delineated differently. Secondly, genealogy is a living discipline. Standards change and new standards are introduced. Some family historians compile data over decades. Not all software updates at the same time. Consider those challenges in the face of GEDCOM 5.5--it wrote helpful guidance into QUAY and the guidance became anything but helpful. As one example, 5.5 QUAY directed users to consider "preponderance" of the evidence"--but BCG subsequently raised the standard and removed that specific guidance from the GPS. Does that example make my point?

Separately ...I didn't think this Source 01 was ready to move to the Developers' Meeting agenda, because I think we need more curent documentation.

In particular, Tom Jones Inferential Genealogy (linked on the wiki, but you can Google for it, too) he writes, "Consider both indirect and direct evidence, including negative and circumstantial evidence." I didn't want to nuance that passage into a standard without reasoning it through a little more.

Summary: Mills concept of very clear check boxes made a lot of sense to me as a "way forward."

I personaly do not realy understand what some of these classifications is used for. Are they output in reports? In most cases I would know these classifications from knowing the source. But, leave that aside, as it seems that there is some interest in having classification schemes.

If we are going to have this in the standard, we should try to arrive at agreed values - leaving this to implementors or allowing several shemes will create chaos.

There have been some proposals above to break things down into several consepts that classify a single simple (atomic) aspect, I support that approach. The simple facts are less likely to change over time. How vendors map (combine) theese in the user interface, if at all, that is their problem.

We should NOT create schemes (ala QUAY) where an increasing value 0-1-2-3 indicates a better quality. We should allow a extensibility by allowing eg. a code value 5 to mean e.g. something betwen 1 and 2. The user should relate to a definition, not the code value.

But, we may need to keep QUAY for backwards compatibility - if it's values can not be maped to a more detailed multi value scheme - but then deprecate it's use when recording new data.

Please see Source02 and the discussion for QUAY.

It would be helpful if the comments about QUAY were held in the discussion that has already been developed for that requirement, Source02.

Separately, I was logged out again, so my response to the balance of TestUsers questions has been lost. Too much to do to replace.

I'll wait to see how the responses develop.

Seems like there is a flaw in the implementation of the wiki, it should offer you an option to log in when you try to post.

Adrian, that was my addition also.

I can't say I've ever run across tertiary in use in genealogy. I was strictly using natural language grammar (or at least European grammar).
First person, second person, third person. I have seen Primary , secondary and tertiary sources used, but can't say in genealogy or not.

Here's my glossary for it.

Primary:: any direct witness or document to an event/object (yes that must allow for the inclusion of the mother 30 years later). I trust if I were to get shot at 3:54PM, and lived another thirty years, I bet I'd be able to recall accurately that I was shot on 22 Feb 2011 at 3:54 in the afternoon. I can't imagine a mother giving birth is any less memorable. ;')

Does this also mean that my data entry in my genealogy file, will become secondary if I live another thirty years? Or does the act of recording it negate that. I speak strictly of the things I have direct knowledge of. My birth, which is more than thirty years past now, as well as my marriages, divorce, child and her stats, etc.
My first marriage is also running up on that magical 30. Now I wouldn't trust any dates my father gave me, but he can't even remember me now. How do you judge what is and isn't believable. My grandmother had my mother convinced she was born in Paris, which is more than a pebble's throw from Hericourt, Alsace. But there was ample evidence to suspect the verity of anything my grandmother said. Even when she wasn't speaking Alsatian.

Secondary: any record or source once removed from primary. So a baptism record indicating legitimate birth could be secondary for a birthdate and marriage of the parents. As could any transcription of an original record.

Tertiary: any record or source further removed than secondary. Such as an article that used a secondary source (a copy of a copy). An article that quotes a book that quotes another that claims it used an original. A cousin's recollection of what he was told by his aunt about his cousin's birth.

Why there seems to be a lack of this is a mystery to me. It probably doesn't matter much but might be useful. I would certainly use it. I can't imagine, I'm that unique. But then I do get a lot of "you're weird man". ;')

I'd add "speculative" to that list. I have a small number of records with notes attached simply saying that, along with "todo" tags.

I'd support this though I worry about 2 aspects:
- firstly whether we would end up with "employed as a signaller (certainly) in 1881 (certainly) in Crewe (probably) by the North Staffordshire Railway (possibly)"
Or would it make sense to keep one level of confidence at the overall conclusion level? (e.g. "possibly employed as a signaller in 1881 in Crewe by the North Staffordshire Railway" with notes explaining that "his occupation in April 1881 was signaller and he lived in Crewe - since he'd previously been employed by the NSR, he might still be working for them - in any case his place of work was likely to be close to Crewe")

- I am concerned that some of the words could be interpreted differently. "probably" and "likely" mean the same to me (probability over 50%). "possibly" and "perhaps" mean the same to me (probability less than 50%). Not sure what "apparently" means... I do like being "precisely imprecise" but unless we put limits on, we might end up being "imprecisely imprecise"
To me there are only 4 values:
- 95% to 100% (e.g. certainly)
- from 50% to 95% (e.g. probably)
- from 5% to 50% (e.g. possibly)
- from 0% to 5% (e.g. unlikely)
Brian's "speculative" feels to me useful but different - it implies "unknown probability".

Hi Adrian,

In your second concern, you wrote, "I am concerned that some of the words could be interpreted differently."

I agree we won't don't all interpret or use the terms in the same way.

Use of these levels of confidence indicators begs the proof (by footnote, proof argument or parenthetical comment).

As a user, I'm particularly interest in seeing terms, rather than assigning a numeric value to terms.* Reason: (a) The concept of levels of confidence in conclusions may be new to the GEDCOM world, but it's not new to the genealogical world. We find these kinds of confidence indicators as terminology in evidence genealogists access day to day. (b) In use (function), these are powerful words--there IS a material difference between "probably" and "possibly." Regardless of differences in interpretation, to me comparing "48" to "51" is far less powerful than the difference in saying "probably" vs "possibly."

In your first concern, you wrote, "... signaller (certainly) in 1881 (certainly) in Crewe (probably) by the North Staffordshire Railway (possibly)..." vs "one level of confidence at the overall conclusion level? (e.g. 'possibly employed as a signaller in 1881 in Crewe by the North Staffordshire Railway' ...)"

To me, you are getting at here is the manner and how well one uses available tools to craft and present a conclusion.

Outside of the GEDCOM world, lacking parenthetical comment, I'd look to where the reference note was placed, and assume that referenced note addressed the preceding statement/conclusion. That same flexibility isn't so available in todays software-GEDCOM context.

At this stage, is it possible to agree the use of levels of confidence qualifiers begs parenthetical comment, note (footnote/endnote) or proof argument, leaving the "how we make that related function happen," as a dependency?

-GJ











*How this works, e.g., the terms, the translation of terms, opportunity for parenthetical comment, and whether the field should be limited to specific terms, all to be vetted still.

Adrian,

Yes, that is how I apply speculative. Anything that I think could be true, but I can't as yet provide any facts to support. Like two men with the same unusual last name and children who share a range of common names, speculatively being brothers/cousins.

Gene - just to be clear - I wouldn't ever advocate putting the numbers in. I only wrote them there to provide a quantitative definition of what I thought the words meant. And yes, there is definitely a difference between "probably" and "possibly" that is a lot more meaningful than the difference between 49 and 51!

Yes, I think we have to leave for the moment the idea of exactly which, where and how we qualify. So long as we realise when we get there that because we are coding stuff up, the more sophisticated it gets, the more complex it gets - usually.

Yes. Perfect.

(1) References from the original discussion.

Hoff and Leclerc, _Genealogical Writing in the 21st Century_ (2006), p. 2, "The process of expressing our findings in writing--including proper use of terms such as probably, possibly, likely, and maybe--is the most valuable tool in our research kits. Unfortunately, it is also the most neglected."

Same source, p. 115, "Commonly used Symbols," for "?" as, "uncertain interpretation of original text."

The discussion morphed into other topics, but the link is below.

ref: http://bettergedcom.wikispaces.com/message/view/BetterGEDCOM+Comparisons/32431138

(2) I have seen the qualifier before information and after, but context is important here.

(a) Before ...
(i) _New England Historical and Genealogical Register_ 165 (Jan 2011): 41, for child list entry about "vii. ?DAUGHTER, bp. Copford 8 Feb. 169/20"; same treatment, also in the child list at p. 48 for a ?SON and a ?DAUGHTER.
(ii) Also before in NEHGR July 2010, p. 187 for the child list entries about ?Ebenezer Handy and ?Hannah Handy. ... "ii. ?HANNAH HANDY. As mentioned above, it is not clear whether Hannah Handy, one of the grantors in the deed of 1 July 1802 given above, was the sister of Silvanus Handy or his mother (or stepmother)."
(iii) Will have to track the third down, was the question symbol in front of the child's listed number in the child list.

(b) ...and also following information, but when it follows, appears in editorial brackets (the brackets also protect the symbol from being confused with punctuation).
(i) _New England Historic and Genealogical Register_ 165 (Jan 2011): 30, for "...and another daughter Phebe was born to them on 4 February 1732[3?] ..."
(ii) Quite frequently in NEHGR to indicate that whether a date should have been a double-date is unknown. For example, NEHGR 163 (April 2009):107, twice, "1680[/1?]" and "1740[/1?]" ... NEHGR 164 (October 2010): 283, 290 for "9 March 1677 [1676/7?]," and "15 Feb. 1742[/3?]"
(iii) For part of a date ...NEHGR 163 (April 2009):139, "400 acres in Killingly on 5[?] January 1718/9" ...

Note: I think an unbracketed question mark symbol that follows an entry would be too easy to confuse with punctuation.

(3) NGS has transcription standards. I haven't looked at those in a little while. Will have to first check BCG Genealogical Standards Manual.

(4) Again not directly related to this requirement ... NGS Quarterly uses dashes to signify a missing name, and NEHGR uses underscores. If I'm not mistaken, four of each (dashes or underscores). NEGHR uses the same underscores for "missing information" [Hoff and Leclerc (2006), p. 116 (Appendix B)]

(5) Similar but different, see requirement "ConfAcc02 (was Data04)":

April 2011 NEHGR, page 125 has entry in the child list for "ii. _probably_ JOHN PYLE, infant, bur ..." and the same term, same treatment for four children in the child list on page 130; again on p. 133.

In the earlier October 2010 issue (NEHGR), p. 264 has entry "(perhaps)" in the child list, as "iv. (perhaps) DAVID WATERBURY ..." same style but "(probably)" on the next page for an "unidentified daughter."

Oh Gene - is that sound I hear, the sound of a can of worms being opened?

I'm not sure how many different combinations of characters and words we seem to be creating that describe uncertainty. And if the learned societies can't agree or be consistent (or am I being pessimistic?) I'm not sure what we do...

But I'm not sure how much we need to do.

If someone enters ?Ebenezer Handy into their genealogy pgm, then I _assume_ it enters the database as
first name ?Ebenezer
surname Handy
This will then get processed as if ?Ebenezer were a normal name - BUT it will not get sorted next to Ebenezer. If you want that to happen, then you either swap to use Ebenezer? Handy or you store it as
uncertainty-prefix ?
first name Ebenezer
surname Handy

The issue of dubious dates is more complex since a lot of software expects dates to look like, err, dates. Certainly in the s/ware I use, if I write "15 Feb. 1742[/3?]", then it won't get processed as a date but as a piece of text with no date meaning. If you want it to appear as a date according to some arbitrary decision, then there are 2 options:
- enter it as string "15 Feb. 1742[/3?]" but tell the database to store an interpreted date of 15 Feb. 1742. Or 15 Feb. 1743 if you wanted.
- create some new uncertainty rating of "Unsure if OS or NS date"

Personally, I'd go for the minimum change and use the facility found in some s/ware of a text string (i.e. "15 Feb. 1742[/3?]") having a date interpretation added to it.

A linked issue is that of transcription markings. I always use square brackets [] round unclear text and angle brackets <> round inserted text. But I've no recollection where I got that practice from. Worse, we have people using [] for both purposes, so I've no real idea whether when they write "[sig] John Doe", do they mean John Doe signed this? Or do they mean the three letters "sig" probably appear in front of the words "John Doe"?

HOWEVER - I think we just ignore this debate in BG and pass forward whatever we get. I don't _think_ we need worry about these transcription standards. Do we?

Hi there Adrian:

(1) My personal preference is to allow either the symbol (?) preceding the entry or the symbol in editorial brackets that sets the entry off with specificity [/12?]

(2) As in the original thread ... I was hoping genealogical technology standards could bring us the form of a preceding symbol that would be neutral for sorting purposes.

Generally, the requirement here (universal qualifier) if part of a the broader discussion in the original thread, all of which recognizes the need to move away from absolutes and into the real world of genealogy.

Original thread suggests we all want this, so perhaps we keep collecting ideas and input for now. --GJ

I wonder if this is more about the software than about the data. A program that displays the information to the end user can put a ? at the start or the end of a piece of data. It could also display the information in a different font, a different colour or whatever.
Likewise for data entry - a program could use whatever interface it wished to permit a user to specify that information is uncertain.
But the way that information is stored in a file could be quite different. While a ? might suit the end user in many cases, that isn't really a matter for BG to specify.

Hi Redmanvan,

The question might be, do genealogists need a standard way of communicating "I'm not quite sure, but I think ..."

If you can't make out a record, you might enter an underscore, "_", but what if you think you can read the name.

This morning I quoted from a text in which the brides name had been published in 1894 as "Mary Witherton [?]"; so the practice of not being able to decipher a name but not being quite certain.

What do you think?

*"so the practice of being not quite certain, but practically certain, is not so new.

I agree that a standard way of communicating uncertainty is useful - whether a simple question mark is sufficient for every case is arguable.

But my point is, the meaning ("This name is uncertain") can be expressed in so many ways, that we should not force BG to use a ? mark in its internal representation of that meaning.

In fact, in the example you quote, it may convey uncertainty, but it certainly doesn't convey indecipherability.

If BG is to convey uncertainty, it should also convey the reason for that uncertainty, and if there is more than one possible reason (indecipherabilty, inconsistency between records, unlikely spelling, etc.) than a ? is not enough.

Alyn

HI Alyn,

Aside from the need to provide the rationale, is there a reason you would not want to provide some symbolism? (Whether it was a question mark or a smily face.)

I agree that there are various ways to convey uncertainty and that indecipherability is only one reason for uncertainty.

"If BG is to convey ... it should also convey the reason ..."*

I didn't mean to suggest that the whole would not explain the reasoning, but there should be a way to convey what has been separately explained.

In my case, that explanation would usually be found in a citation, but I still have the other databits to be concerned about. Another example follows.

Today, I confidently write the name of one of my ancestors as Elizabeth (Clark) Preston. For many years, though, the best evidence I had about her maiden name was a historical marriage record containing modifications. Her married surname had been clearly entered to the record. By some different pen and/or hand, that surname had been crossed out and the surname "Clark" had been written below. (It's possible that the changes had been entered and then erased.) The record I'm referring to can be viewed here:
https://familysearch.org/pal:/MM9.3.1/TH-266-11124-173590-21?cc=1520640&wc=7131654

I can easily record a citation to that record in which I can explain the issues; have even spoken to the town clerk who holds the record and have learn more about it.

During the time when that marriage record was the best evidence of her name, I might have entered Clark[?], associated with the citation I described. For my purpose, that entry would have been preferable to leaving the surname blank.

How the uncertain information is handled involves judgement. In other words, what I might feel is worthy of entry, albeit uncertain and cited, another might see as unworthy of entry at all, only commented upon in the citation. This requirement wasn't intended to set a recording threshold, just to provide a way forward when the genealogist feels it has been met.

(I suspect many genealogists already use the symbol ?, so as Adrian suggests above, we pass it on. I'm suggesting there is probably a benefit to recognizing it and encouraging its use when folks hit that threshold. As long as BG is not being stripped off or relegating the associated data to as "irregular," there are probably other ways of promoting the use of the symbolism.)

*Recognizing that BG can only convey that which it has been "fed."

This is an area where formalisation may need to be justified. I know a lot of transcription groups have a syntax for representing unknown characters or characters that may be one of a set. www.freebmd.org.uk is probably the one I am most familiar with. Those syntaxes - which are generally some form of "regular expression" - are easy for software to read, and to work from, but many users would have to look in a book to see what it means.

My question is therefore whether this formalisation is for the benefit of software or end-users.

I thought about this when designing STEMMA and decided to defer any decision. At the moment, I am recording the transcription as best I can (i.e. most obvious and/or most likely), and using STEMMA's narrative feature to record why or how it may be uncertain. That actually works well from an end-user point of view but any name matching algorithm would have nothing to work from

Tony,

It's probably useful in the "data" field of source references. But I don't see it needed anywhere else.

Louis

Hi Louis,

The purpose of this request was to have better support in the pfact data fields. Those fields are used for other than just tree matching. In the programs I use, they are part of the sentence and narrative structure.

Narrative should be searchable too. This is why I have tried to push STEMMA's concept of "structured narrative". In principle, its mark-up language could record the "regular expression" syntax in such a way that it does not detract from the visible text being read by the end user (...in the same way that a URL link shows you the link title without the link syntax).

Agree.
I've never read any Genealogy standards book (They seem to be much more common and important in the US), but I kind of came up with the exact same use for the "?". Though I also used it for showing "uncertainty" anywhere my software doesn't allow a "surety", e.g. with dates. Which in turn makes my software complain about invalid dates...

I should finish my morning wake up before trying to respond.

The trick is we want software to be able to recognize that symbol but also include it and ignore it for the purpose of generating lists or setting things in date order. So, 11 ?Jan 1837 would sort just after 10 Jan 1837, and before 12 Jan 1827; John ?Williams would sort before Williams, Johnny, but after Williams, Jan.

Yes, exactly. My current software is to stupid for that ;)

"11 ?Jan 1837 would sort just after 10 Jan 1837" - that's probably not a problem. At least two ways of doing it that I can see:
- store both "11 ?Jan 1837" and "11 Jan 1837", the first as the real date, the second as the date to be used for sorting, arithmetic, etc
- store "11 ?Jan 1837" in bits (i.e. "11", "?", "Jan", "1837") and recreate the date for sorting and arithmetic from the "11", "Jan" and "1837" bits.

That is great news! tyty

Adrian wrote:

"11 ?Jan 1837 would sort just after 10 Jan 1837"

to me this is saying

I know it was the 11th
I know it was 1837
It was possibly (or I think it was) January

so I'm not sure why it would be made to sort after 10 Jan 1837.

If there is to be a qualifier like this, it should be attached to the element it's qualifying, which in Adrian's example would be the 11 not the Jan part of it, so

?11 Jan 1837

@Kiwi,

See the first presentation of the example (third entry in the discussion), where "Jan" is the element questioned.

I am not sure that I see any advantage in encoding this as a separate bit along with bits for day, month etc. I think I would like to see the ? in a string encoded date, most likely accompanied by an optional sort date.

But, I would suggest that a "surety" value, a separate bit, meaning ?, could be attached to the whole date, possibly to one of the two dates defining an interval or a single complete date.

I have already suggested to have a surety value attached to the link between levels in a place/location hierarchy, and there should be surety attached to the link from an event to a place name (thus possibly putting a question mark against the whole path from the bottom level place name to the one on the top level).

And it should be possible to attach ?s against all sorts of relations - between persons, names, even media. I would love to see a ? against a line between persons in a chart.

In the same way as ?, I would in some cases want to see a character indicating dissproval.

I should also mention that in national standards here for transcription of church records and census records, double question marks, ??, are used to indicate that the source is difficult to read so instead of January it could say Jan??y. ?? are used because single a ? may appear in the source. I have seen many dates in church records containing a question mark. (If there are two possible interpretations of a word we write both with @ between, !! marks missing data or an obvious error in the data, and %xx% means xx has been crossed out.)

Why not put the "?" after the part that you can't read, then the sort will come out close to where you want it without any extra work?

This not only applies to dates which have a small range of possible values, but names and locations that are hard to decipher or were spelled phonetically.

gthorud makes a good point here: "I should also mention that in national standards here for transcription of church records and census records, double question marks, ??, are used to indicate that the source is difficult to read so instead of January it could say Jan??y. ?? are used because single a ? may appear in the source. I have seen many dates in church records containing a question mark. (If there are two possible interpretations of a word we write both with @ between, !! marks missing data or an obvious error in the data, and %xx% means xx has been crossed out.)" I think the "Way Forward?:" involves research to see what else is commonly done when text is unreadable.

I would agree with this. There are clearly researchers who use the evidence-conclusion model. I would even go as far as saying we are all using the evidence conclusion model, and it would be nice to have a place to put my thoughts down on things I can't "prove" 100% with primary sources.

Agree, too.
IIRC, there have been a few discussions that touched on a possible way HOW to do it. I tried to come up with something in this message.

This requirement is less related to the model than the process.

It's sometimes difficult to incorporate a proof argument in genealogical software because the written proof must be related to more than one source; often to varied facts and even facts about more than one person.

"It's sometimes difficult to incorporate a proof argument in genealogical software because the written proof must be related to more than one source; often to varied facts and even facts about more than one person."

This statement captures one of the main reasons I like the model in which every conclusion person is made up of a tree of person records representing either evidence or previously made conclusions. Each leaf person has its source. And each person made by grouping other person records can have a proof statement based on just the person records being joined (ie, being concluded as being the same real individual) at that decision point. Then the proof statements don't have to justify bringing all evidence together at the same time, which would make an impossibly awkward proof statement. Each proof statement only has to refer to a single decision at a time. Sorry that this concept is a bit difficult to plow through.

TW

As I understand it, the "model" requirement is Evidence01. This Evidence02 requirement is intended to be not dependent upon implementation of Evidence01.

Perhaps we need an Evidence03 that is dependent on Evidence01, with Evidence03 being the proof argument methodology for those who would implement the model in Evidence01?

Gene - I suggest you don't need Evidence03 - at least at this stage of the game. You have a requirement expressed as Evidence02. As you say, you're actually thinking about the process in Evidence02, rather than the model as such (which is more related to Evidence01). I suggest you stick to that and don't try to second guess how Evidence02 might be implemented.

It may turn out to be that there are (say) 2 ways of actually designing a solution to Evidence02
- one that happens to require a data model that's most or even all of the way towards also satisfying Evidence01
- a 2nd way that's only vaguely or not at all helpful towards Evidence01

It's at that point we take a decision which is best. Right now, we don't need to worry about the "how" for Evidence02 so you can leave it as it is.

Make sense?

Hi Adrian ... makes sense to me, yes.

My page on http://bettergedcom.wikispaces.com/Research+Process%2C+Evidence+%26+GPS seems to be useful in that it defines _my_ view of what the steps are and what the data is.

I thought this would be a very useful field. In my idea, a task would start out life as a "todo" task, and when completed it would be a completed task that would document the research done and optionally record the success or failure of the task. Naturally not all tasks are related to direct research or wanted to be kept when completed. So a flag might be nice which when checked is discarded/ignored when completed.

I second this idea.
GenXML has a task, too.
http://lineascope.com/ uses tasks, too.

Anyone think GenXML's Objective would also be useful?

Re: "BetterGEDCOM must allow a "date" to be entered as a phrase where the values are not recognizable to a date parser, but which gives a human reader information about when an event occurred. It must allow such a phrase to have an optional date in parseable format that can be used to interpret the phrase"

See my post of today under Data-Date02.

All determinate date values in BG should be numeric-format ISO 8601. Date-entry and date-formatting should not be relevant to the stored BG data.

However, two areas that are relevant include:

1. Provide a description of the Date that may include additional semantics, e.g. "Christmas 1956". This is separate from the stored date value and could be used as a display token when it appears in Narrative text

e.g.

<Date>
    <Value> 1956-12-25 </Value>
    <Name> Christmas 1956 </Value>
</Date>

1. Support for an undecipherable date. This may need a transcript of the original material, or a citation. Also, maybe a best guess at the date's value together with an assessment of the validity

As in some other posts, I am taking a very long-range view here. Where I see the specification of this requriement with the words "a phrase where the values are not recognizable to a date parser", my thinking is that we need to support the development of better date parsers, so that 10 or 20 years from now, a date parser can read a date that is specified as at the time of some battle and be able to supply standardized dates, either a specific date or a range for that event. And the same with "on a Tuesday in the Spring of 1873", which is a finite set of specific dates.

Certainly we have to live with the reality of the simplistic date parsers of today. But we should be working to move to a future where that is being robustly addressed and supported.

Dates are often not precise in genealogical contexts. Dates can be “about”, “around”, “before,” “after,” “on or before”, “possibly,” “probably,” “computed,” “interpreted,” “estimated,” “between” this “and” that, “from” this “to” that, this “or” that, this “and” that, as text strings (e.g., “in the year his father died,” “around Christmas a few years ago”). In colonial times we have double years to contend with. For older dates we might not know what calendar the date is from. Dates might be in ecclesiastical form or in terms of the reign of a monarch. We find months that are expressed as words, abbreviations, roman numerals, numbers, and goodness gracious, not in English.

In the evidence we use to find our data, dates are written in many ways. When extracting those dates into event or persona records are we willing to loose the original expression of the dates by translating them into a standard form? I think most of us would agree that we must be able to keep the original wording if we choose, even if we try to translate the dates to a standard form.

Why do we need precise, standard dates in genealogical applications? When do we need things accurate to the day? Or week? Or month? When is it critical to know what calendar a date is from?

Most of the time can’t we just think of a date as a tag associated with a person or an event that can be used as a sort key?

How do we use dates in genealogical applications? For putting lists of people in order. For help in searching for persons and events by providing dates for filtering. To show on forms and display on reports. Maybe to compute average life spans. Which of these requirements need the dates to be in a fixed, standard form? Which require the dates to be known to the day? Which require us to know the calendar? If in a long list of persons sorted by some date property, how important is it if two are switched with respect to the unknowable truth?

I have written a short document that explains the approach to dates taken in DeadEnds. The date class in my software has a context free grammar parser that extracts software date objects from any strings that are written in any of the forms as shown in the examples in this document. Here is a link to the document:

http://bartonstreet.com/deadends/DateFormats.pdf

My contention is that computer geeks naturally tend to want precision and strict standards, even in “humanistic” areas that really can’t support that level of precision. I feel the same way about trying to standardize places to a simple latitude and longitude pair.

Re: "As in some other posts, I am taking a very long-range view here. Where I see the specification of this requriement with the words "a phrase where the values are not recognizable to a date parser", my thinking is that we need to support the development of better date parsers, so that 10 or 20 years from now, a date parser can read a date that is specified as at the time of some battle and be able to supply standardized dates, either a specific date or a range for that event. And the same with "on a Tuesday in the Spring of 1873", which is a finite set of specific dates"

As I said in Data-Date02, no such beast exists and it is not possible in the general case Wesley.

The transcribed date string could be retained, obviously, but the interpreted date value should be determined by a person rather than some flakey date-parser with no well-defined grammar. [I was a compiler writer for over 10 years]

Re: "Dates are often not precise in genealogical contexts....."

You're misunderstanding me Tom. Error margins and relational qualifications can still be handled with ISO dates. Also, retaining the original transcription is not being questioned here.

The issue is how to store machine readable dates when those dates (including the margins) have been determined or estimated.

You've already made a good point about the formats and semantics being indefinitely vague. Hence, no date-parser software is going to do it for you.

We're talking about a storage and exchange format here - something that "computer geeks" know all about. I'm definitely not suggesting we throw away evidence or transcriptions. :-)

I've seen so many misguided projects where someone has decided they can reverse-engineer a print-format date, or a print-format currency, to get back to a machine-readable version. It just cannot be done in the general case!

When dates are in other calendars then there are other, more serious problems. Some do not allow an accurate conversion to a Gregorian one. I've a vague recollection of other international standards that apply to such dates but I can't find the document at the moment - I need a strong coffee and a torch.

re ACProctor: "As I said in Data-Date02, no such beast exists and it is not possible in the general case Wesley."

No such beast exists today, and a perfect one will never exist, but that does not mean that we should not be moving toward implementing one that is constantly improving.

We should not restrict our long-term vision to that which the technology of today limits us. I see BetterGEDCOM as an organization that is on-going, not just to create a product and then disband. As technology evolves, the need for new versions and associated supporting mechanisms, such as places and dates, will improve. We should have a long-term vision of what we would like to see happen and then gradually make that vision happen, even if the current state-of-the-art makes it seem that is never going to be possible.

Look at where we are today with optical character recognition -- which is like "magic" compared to where the technology was just 25 years ago. The US Post Office is even using it to sort hand-written mail. It is not perfect, but it can do the vast majority of the job right.

The ability to do things 25 years from now is something that we cannot conceive as possible right now. But we can still set a long-term vision and as technology progresses we can expect that that vision will be more and more realizable.

It's not a case of better technology so much as simply not being practical is most cases, and totally impossible in others Wesley.

A simple look at database storage will show that they don't store and manipulate date strings - they store and manipulate compiled versions (well, in well-designed software anyway). If software has determined a date, or a rigorous researcher has deciphered one themselves, then all I'm saying is that the machine-readable version should be stored in one standard format, not one of a limited set of accepted print formats. It makes no sense to keep decoding the same strings over and over again each time the data is loaded. In fact, there would be no guarantee that you'd have the same date if someone else was using a different implementation of the parser.

I have admit to being surprised that you would entrust some flakey date-parser to effectively generate "facts" in your data anyway. It feels like entrusting an OCR to transcribe a birth cert and simply accepting it blindly.

On the subject of OCRs, have you looked at the results of the recent British Library and Bright Solid venture digitising newspapers? - they're really awful with loads of errors.

re ACProctor: "entrust some flakey date-parser to effectively generate "facts" in your data"

I guess I see now where we appeared to differ. I am not viewing these tools as replacing my thinking; I am not entrusting them to generate anything in my data without my review of their results -- which could be a single result or a small set of results.

I see these as advisory tools, for the same reasons that I do not expect Google Translate in today's world to generate a perfect translation, so that if it is a language with which I am familiar, I will use the translation as the starting point and modify it where I spot problems -- not that the result of that modification will be perfect either.

The output of a date parser, which I do not think will be flakey if there is a good effort put into it over the next 10 years, is a source just like any other, to be weighed in conjunction with the other sources.

Advisory systems are a technology already in use in industry and medicine, aimed at narrowing down the choices. I really do see a robust -- though not perfect -- date parser as a very realizable application over the next 10 years, if there is sufficient support for it. And just as with all technologies, it will continue to improve as more iterations resolve more problems.

[Tirade on]
Another opportunity for me to go off on my "humanistic versus scientific" data tirade. Humanistic data has a high degree of uncertainty about it. That uncertainty is part of the data. It must be loved just as reverently as we love everything else about the humanistic, genealogical data we collect. Trying to hide the uncertainty under the guise of some crazy-looking precise-looking formatting does an incredible disservice to the data and genealogy as a whole. (Places are uncertain entities; names are uncertain entities; we must embrace their uncertainty as well).

Any date format that requires the computation of the number of days for its Better GEDCOM representation is way off base. I would suggest that one might want to read about GEDCOM's date format or my own DeadEnds data format, once more, which is at

http://bartonstreet.com/deadends/DateFormats.pdf

Yes, these formats are Western-culture-centric, and they can be extended with other calendric forms.

If you know that you had a Dutch ancestor born in April 1633, isn't that enough for you? Yeah, it would be great to know the day of the month, but just knowing the month is wonderfully incredible. What advantage do you get from converting that into the number of days since, say January, 1, 1000, in the Julian calendar. You'd have to know the actual calendar of the date (it could have been Julian or Gregorian), you'd have to know the the actual day of the month. Since you don't know these things it's just plain dumb to convert this into a number of days. Just read Adrian's example to see how dumb.

If you do need to sort that date with others you can just assume the day is in the in the middle of the month to get the best statistical sorting out the uncertainty. Isn't this more than enough? I am always amazed how computer geeks want to complexify data to make it seem more scientific. Genealogical dates are not scientific dates; they are sloppy humanistic dates, and we must deal with them exactly as they are.
[Tirade off]

Could you give another example of something that is done more than one way in existing genealogy programs (besides in-line notes vs note-records, which may be a terminology issue)?

Being able to go backwards should not be a requirement. By incorporating all the data features of existing genealogy programs into BetterGEDCOM, by definition, all of the BetterGEDCOM data won't be able to be incorporated back into those existing applications since different programs have different features. However, the BetterGEDCOM data will be able to be imported into future versions of those programs after they become Better-GEDCOM-qualified.

Hi Christine.

This Syntax06 was a posted "goal" moved over from "The original [BetterGEDCOM] goals page," and has been referred to elsewhere as "current goal 7." Unfortunately there was a former goal 7 that had nothing to do with Syntax06.

The original goals page is located here:
http://bettergedcom.wikispaces.com/GOALS

I check the History tab to the original goals page. Appears Louis Kessler added this goal Dec 29, 2010 7:39 pm. I'll try to link the history page here, but not sure it will take.
http://bettergedcom.wikispaces.com/page/diff/GOALS/190247292

There was a lengthy discussion about the original goal. See the discussion here ...
"Single way (current goal 7)"
http://bettergedcom.wikispaces.com/message/view/GOALS/32130746

Hope this helps. --GJopefully Louis will comment with some examples for us.

I believe that the name entity needs to be contained in multiple fields which would include at least NameType {Legal, Birth, PriorName, MarriedName, NickName, StageName, PenName, Alias ... (all defined unambiguously)}, Salutation, LastName, FirstName, SecondName, ThirdName, ForthName, Postfix, LastNameSortField, FirstNameSortField... IF FirstName, SecondName, ThirdName, ForthName is not acceptable, there must be some standard technique to easily separate them, perhaps using a comma and space or just a space but compund names would be joined by and underscore. These name components should be capable of being reference to a set of domains and these domains should be naturally exchangeable via the BetterGEDCOM.

Gier says, "I don’t understand why there must be Level One Vital Events, why can’t they all be Level Zero True Events?"

They could be all level zero. But think of it this way. You get birth data from many sources. Think about getting an age on a census record. This is a great example. You subtract the age from the date of the census and you get an estimated birth year for the person. The census might also list the birth place of the person. So from the census record you have an estimated birth year and a possible birth place. BUT, BUT, BUT, you never really got an actual birth event for the the person did you? You didn't find real hard evidence. It's all secondary information extracted from the census event.

So, one of my principles about genealogy is that you create event records from the evidence you find about events, and you create person records for all persons mentioned in the events, and you add to those person records everything you learn about the persons from the event evidence. In my mind it is much better to think about that inferred birth information as something you learned about the person from the event, so it is something that should be kept inside that person. That's my principle again. It's a compromise of course, as all things are.

See, I don't mind having two ways to do something, if it makes sense to me to have those two ways. In the case of the vital event and the multi-role event I definitely see a difference, and definitely feel it's okay to treat them differently. But every vital event could of course be transferred into a level zero event record if there were a rule that said that was the only way events could exist in the file format.

So for me a vital event, a level one PFACT-like thing inside a person, is just the right thing for these secondary, inferred, not quite events, that we often learn about people offhand through records that were really created for an entirely different reason.

Hope this isn't too confusing.

And of course, there is a very practical answer to the question as well. When converting GEDCOM data to Better GEDCOM format, if there were no level 1 vital events, we would probably triple the size of the resulting file in terms of records and maybe size!

Tom W.

Geir - Re "the structure that some people currently use to transfer hair color, caste, eye colour, nationality etc. although some of this info may change over time – something the Trabant can’t transport"
In GEDCOM 5.5 (at least) INDIVIDUAL_ATTRIBUTE_STRUCTURE (which includes physical descriptions, so I hope that's what we're talking about) includes an EVENT_DETAIL and that in turn includes a DATE_VALUE - so the GEDCOM attribute (the Trabant (grin)) should be able to transport dates from 1 person to another.

A person's name, however, does not have a date in GEDCOM 5.5, giving the impression (to me at least) that all names on file for this person, can apply at once. Name (and sex) are not, in GEDCOM 5.5, attributes in the formal GEDCOM sense of the term.

It is pretty clear to me that Name (and Sex) should be included in the attributes, or whatever we end up calling them, in BG, and that all "attributes" in BG should have dates (or date ranges) in BG (a) because they need them and (b) because most of them could have them now.

Now whether people want to use the dates, whether (if they do) they want to stick a NAME-CHANGE event (say) chronologically between 2 NAME attribute values, is entirely up to them. We just have to provide the facility for those who want to do so.

Adrian,

Your latest about regularizing name and sex and every other PFACT type of thing seems to me to be the approach Better GEDCOM should take towards a generalized attribute concept.

I think the vital event, if we decide to keep it, fits in this category also.

Okay then, there is now a NEXT CONCEPT that needs discussion in the attribute vs event context.

Gedcom has the ASSO tag and DeadEnds has the relation structure and I'm sure other formats have other constructs for the same concept. I have even seen the tags FATH and MOTH in some GEDCOM files.

What I am talking about is an "attribute" that is basically a pointer to another record with a label on it to define the type of the relationship. Note that in the flurry to get rid of the GEDCOM FAM record, such relationship pointers, or a concept that allows the recording of the same information, becomes paramount -- without them genealogical databases would hold no relationships.

It boils down to this question -- IN BETTER GEDCOM HOW DO WE WANT TO EXPRESS THE FACT THAT PERSON A IS THE FATHER OF PERSON B?? I'm assuming that the anti-FAM people have their way and we have no FAM record to help us out. Without a FAM record what will our world be like? Please realize something very important -- in today's world the FAM RECORD IN GEDCOM HOLDS 99.99999% of all relationship information between people. With no FAM entity ALL THIS RELATIONSHIP INFO HAS TO MOVE SOMEWHERE ELSE.

If you check out all the models and what people have written you'll find these three answers:

1. Relationship objects -- this is a common answer. Create a new record type (if you love relational databases you'd call it a new table type). A relationship has a type (parent/child, brother/sister, etc) and then pointers to the two persons in the relationship. Lots of similarities to the concept of the multi-role person, but not identical. As a RDBMS table it's three columns, a type and two foreign keys to a person table.

2. Assertion objects -- this is the scary answer -- a la GenTech, where EVERY relationship between EVERY RECORD TYPE must be mediated by a different assertion object. Assertion objects were also created because they have a simple direct implementation as a RDBMS table. In reality an assertion is nothing more than a generalized relationship -- this is required in the GenTech model because EVERY RELATIONSHIP BETWEEN EVERY KIND OF OBJECT MUST BE IMPLEMENTED AS AN ASSERTION (EVEN RELATIONSHIPS BETWEEN ASSERTIONS!!) -- to go off onto the anti-GenTech tirade for a moment, I hope you realize that in the GenTech model, EVERY PHACT (BE SURE YOU UNDERSTAND WHAT THIS MEANS -- EVERY, EVERY PFACT) IS ALSO ITS OWN RECORD AND FOR YOU TO ADD A PFACT TO A PERSON YOU HAVE TO CREATE THAT PFACT AS A SEPARATE RECORD AND THEN CREATE A NEW ASSERTION RECORD TO BIND THAT PFACT TO ITS PERSON -- "Help, help, I've fallen and I can't get up!"

3. Direct references -- person B points to person A with a pointer that implies "You're my daddy", and maybe vice versa.

We're going to have to pick one of these for Better GEDCOM. OpenGen has been sniffing around the relationship approach. My bet is that SoRD is sniffing around the assertion approach. In DeadEnds I've opted for the direct reference approach. My reasoning has been that the relationship and assertion approaches both require the addition of (at least one) new entity type to the model as well as a large increase in the number of records in actual external files. In the direct reference approach no new entity type is required as each person in a relationship simply points to the other (it could even be others).

So what's the relationship between this discussion and the discussion of what is an attribute? It' pretty simple really. These direct references that could be used to establish relationships between persons also LOOK, ACT, SMELL, and so on, like all other things in our "extended" view of what an attribute is.

Okay, where can this discussion go after we resolve this one?

Well, we could discuss whether a source is an attribute of a person record.

And my favorite, how should we treat the evidence persons that provide the grist for the conclusion mill, as components of a conclusion person? That is, ARE THE EVIDENCE PERSONS THAT PROVIDE THE DATA BEHIND A CONCLUSION PERSON ATTRIBUTES OF THAT CONCLUSION PERSON? If you consult the DeadEnds model you will see that each person record can contain an unbounded number of references to other person records. In the DeadEnds model this is my implementation of the evidence and conclusion objects. So it's very easy to simply to view these evidence person references as "just another" type of attribute.

I take a basic, data structure view on the whole thing. A data structure is a tuple of information. Each element of the tuple can be simple or a self-contained data structure of its own, or it can be a pointer outside of the current data structure to another data structure. All our entity types in the Better GEDCOM model and in every other model can be thought of as one of these data structures. It is this view that is the lowest common denominator for all other views.

Tom

Excellent analysis about relationships, Tom. That info deserves to be promoted to some place on the site it will not get lost.

One consideration for deciding what to do with this in BetterGEDCOM. We will want developers to adopt BetterGEDCOM. I expect all 500 plus programs out there now use the FAM relationship object. They will NOT change their internal data structures to accomodate BetterGEDCOM, so they will need an easy way to translate from whatever BG has to their internal structure. If that is going to be too much work on their part, they either won't do it, or they'll do it while raising a big stink about it and that won't be pretty because when asked, they'll tell people that BG sucks.

It appears that that it is difficult to agree what we are trying to find a term for. It appears that Tom is defining an Attribute as something much wider that my Trabant, and what I understand was meant to be defined in the beginning of this topic. Tom's definition is more like an attribute as used in a data model. Back to square one.

---

I will come back to other issues, but can someone explain why there can be an event subordinate to what I would call an attribute in INDIVIDUAL_ATTRIBUTE_STRUCTURE in Gedcom 5.5(.1)

Sorry about the last question, I should have read the previous entries.

Tom,

Regarding Level zero and Vital Level one events.

I understand what you are saying, but I still don’t see why there could not be only level zero events. And I don’t understand why that would triple the size of the file.

---

Adrian,

I have a problem understanding the purpose of the INDIVIDUAL_ATTRIBUTE_STRUCTURE containing an event. Cant find anything about why in the Gedcom spec, have I missed anything. WHY NOT JUST USE AN EVENT – is this a construct that that has appeared after someone has discovered that hear colour can change over time? Maybe someone can explain why there is an event inside this structure?

Also, I am skeptic to having date attached to a name. Isn’t a reference to an event where the name is used enough, without having the date within the same structure as the name? Where would the data come from if not from a source that could be identified an event?

Geir,

I didn't mean to throw a spanner into your works. There may be a few more vehicles out in the parking lot!!

I am really a very one-dimensional person. I see data models in very simple terms, and those terms are nearly 100% simple computer record structures with a little bit of object-oriented frosting thrown in.

I see every entity as a computer record structure, where a computer record structure is a tuple of name fields. There are only a few kinds of fields, and maybe these fields are the things that Geir is equating to automobiles. Here are some of the possibilities:

1. A field whose value is a small number of discrete values from a special set (sex has m and f; event type tags come from an agreed upon set, and so on; other examples left to the reader).

2. A field with an infinite number of values, but whose values still obey strict syntactic rules -- e.g., names, place, dates.

3. A field whose value is a generic string -- maybe the description of something, a note, a free-format description of a source.

4. A field that has its own internal record structure -- vital events are like this -- a BIRT in an INDI is a "sub-record" of the INDI record -- once you look at the sub-record in its own right, it's just another record structure inside the INDI record structure. Of course, this is the beauty of both the GEDCOM level structure and the XML element structure; you can carry the sub-record structure structure as deeply as you'd like to go. Note that JSON is the same, it's just that JSON, since it comes directly form the need to transport the values of Java objects, is really closely aligned to the idea a record structures (one would expect, once one realizes that the expressive power of GEDCOM, XML, and JSON are all the same, that it would be no surprise to realize that all WE EVER TALK ABOUT IN GENEALOGICAL MODELS is very simple data structuring stuff). It's just clearer in JSON that you really are transporting a computer data structure. And these new fangled things that people are calling "protocol buffers" these days, are really nothing more than these record structures in a binary form that makes them more efficient for moving around the internet through various service APIs, even though they suffer from the fact that they are not human readable.

5. A field that refers to another record -- source pointers are like this -- relationship pointers (if adopted) would be like this -- all this means is that the value of the field is a REFERENCE to ANOTHER RECORD STRUCTURE that is outside of this record structure -- note an interesting point that is rarely mentioned in the genealogical context -- pointers of this type are always assumed to point to the top level of another record structure, that is, to a full record object, what I call a first class citizen, but in fact there are times when you might want to think in terms of a pointer in one record as pointing SOMEWHERE INSIDE another record.

Someone else might break these things up using a different hierarchy, but there really isn't any other major way to see the data structuring world.

From my point of view every one of these five kind of things can be called an attribute because it provides some specific bit of information about the structure that contains it. Should we have five different vehicles for each of these?

In my view a PERSON and a multi-role EVENT are top level record structures, that is they are never found as sub-structures inside other structures. They stand alone; they are their own thing. You can think of them as attributes of the record that points to them even though they are stand along objects. This is what pointers are usually used for in computer programs anyway.

However, VITAL EVENTS, as I've defined them, and as they are defined within GEDCOM 5.5[.1], are always internal sub-record structures and become attributes that way.

Sorry, but my posts seem to slowly turning into "Computer Data Structuers, 101"

Tom W.

Geir - re "INDIVIDUAL_ATTRIBUTE_STRUCTURE containing an event". I think I understand your question but please bear with me if I got it wrong.

In summary - it isn't an event inside the INDIVIDUAL_ATTRIBUTE_STRUCTURE, it's just a load of data that looks exactly like the nearly-corresponding bit of the event does, so rather than define it twice, they used the same "label".

Perhaps if I write it out more completely (I'm quoting from GEDCOM 5.5.1 because the copy action keeps failing when I try to copy bits out of 5.5)

INDIVIDUAL_RECORD is defined in GEDCOM 5.5.1 (page 25 in my copy) as
n @XREF:INDI@ INDI {1:1}
...
+1 <<INDIVIDUAL_EVENT_STRUCTURE>> {0:M}
+1 <<INDIVIDUAL_ATTRIBUTE_STRUCTURE>> {0:M}
...
(where ... as usual means omitted stuff)
So - the Individual record contains 0, 1 or more event structures and 0, 1 or more event structures.

INDIVIDUAL_EVENT_DETAIL is defined as
n <<EVENT_DETAIL>> {1:1}
n AGE <AGE_AT_EVENT> {0:1}
i.e. it's made up of a single EVENT_DETAIL structure, followed by an optional AGE.

The EVENT_DETAIL is defined as (I'm not going to copy the formal GEDCOM out because it turns out I can't copy to the clipboard from 5.5.1 either!)
- an optional TYPE line
- an optional DATE line
- an optional PLACE structure (itself being multiple lines)
- an optional ADDRESS structure (itself being multiple lines)
...

INDIVIDUAL_ATTRIBUTE_STRUCTURE is defined as a whole list of options, virtually all of which follow the same pattern,viz:
n OCCU <OCCUPATION> {1:1}
+1 <<INDIVIDUAL_EVENT_DETAIL>> {1:1}

Yes - the INDIVIDUAL_ATTRIBUTE_STRUCTURE is defined as a line relevant to the attribute plus one INDIVIDUAL_EVENT_DETAIL. The latter is already defined above as
n <<EVENT_DETAIL>> {1:1}
n AGE <AGE_AT_EVENT> {0:1}
and
EVENT_DETAIL is defined as
- an optional TYPE line
- an optional DATE line
- an optional PLACE structure (itself being multiple lines)
- an optional ADDRESS structure (itself being multiple lines)
...

Or in other words, although the standard says INDIVIDUAL_EVENT_DETAIL, it's short-hand for the group of lines consisting of
- an optional TYPE line
- an optional DATE line
- an optional PLACE structure (itself being multiple lines)
- an optional ADDRESS structure (itself being multiple lines)
...
All of which are what you see against an attribute.

So it's not an event "contained in" the attribute, it's a set of lines having exactly the same format as a set of lines that happened to be defined for the event first.

Err - sorry if my short-hand mislead. Does this make it clearer?

Gier says, "Regarding Level zero and Vital Level one events...I understand what you are saying, but I still don’t see why there could not be only level zero events. And I don’t understand why that would triple the size of the file."

Yes, all level one, "vital events" substructures in person and other records can be converted to level zero "event records." They wouldn't triple the size of the file in pure character count, but would probably at least triple the size of the file in terms of number of records. The files would get larger in character counts simply because of all the extra record "boiler plate" and inter-record references that would have to be added.

The vital events as done in GEDCOM now (eg, BIRT, DEAT, MARR, ...) are much more attribute-like than they are event-like. Some might not agree with that. Each vital event describes one vital fact about a person or a family. There are no other role-players. There is obviously an event hidden away behind the vital, but the details of that event were not of concern when the fact was recorded (or if it were, the genealogical application proved inadequate for the task). A vital event is very much like an abstract of a real event, extracting only limited information (generally just date and place) pertinent to the primary role player.

I sense the real concern over dealing with vital events as substructures in other records versus multi-role events in their own right, is that it might seem like we would be sanctioning two ways of doing the same thing. You can see it that way, but I also think that's an incomplete view. Instead of just rejecting the idea of having two ways to do certain things, isn't it better to stop and rationally consider why there are good reasons for having two ways of doing things?

Think about the very practical problem of converting millions of GEDCOM records into Better GEDCOM records. Is there really a compelling reason to convert every BIRT, every DEAT, every RESI, every BURI, every CHR, every MARR substructure in the GEDCOM files into separate single-role event records? There is no technical reason why you can't do it, and if Better GEDCOM does away with the vital concept idea, we would have to do it, but what does it gain?

I've been a software developer for 45 years and a software professional for 40. If there were anybody around who could give advice for or against the idea of whether it's bad to have more than one way to do certain things, it would be me. I find nothing uncomfortable in the notion of having the two kinds of events. My hope is that having the multi-person event record will encourage genealogists to always collect all the info that is available from the evidence. But for the cases where a full-bodied event is not warranted, or there is no real evidence about the event yet, the vital event works well.

Here's another thought. You've just talked to your grandmother to get information about her grandparents. She gave you some birth dates of her grandparents that aren't yet in your database. Are you going to add those birth dates to the records for her grandparents as simply vital events, as would be done in GEDCOM, or are you going to create separate birth records for each of those grandparents. Don't you think it's kind of overkill to create birth events from such distant and secondary data? OK, you can do it.

Also remember that we are discussing getting rid of the family record. If that happens we have to find a new way to indicate relationships between people. Relationships between people are some of the same implied event information that vital events and regular events do. As soon as we face this we will likely encounter a third way that some information can be implied.

My bottom line is that I PREFER a world with both vital event structures and multi-role event records. We CAN'T live in a world with just vital event sub-structures. We COULD live in a world of just multi-role event records. From that it's up to the Better GEDCOM collective wisdom to choose the official path.

Tom W.

First, Thanks to Adrian !!!! for making me look much closer at the Gedcom definitions of INDIVIDUAL_ATTRIBUTE_STRUCTURE and INDIVIDUAL_EVENT_STRUCTURE.
It appears that my Trabant has no motor and must always be accompanied by a Rolls Royce. Based on how Attributes are presented in the user interface of some programs, I thought I knew what an attribute is, just a type value pair, I didn't check the Gedcom. So after checking the Gedcom for these two structures it appears, as has been stated earlier in the discussion, the I_ATTRIBUTE_S is just an I_EVENT_S plus a value (with at list one minor exception, BIRTH). This make a lot of change in my thinking, and I am sorry for the unnecessary discussion I have caused – and it must have been very difficult for those trying to understand what my Trabant was.

(When reading the BNF in the Gedcom standard I note that it might not be necessary to define event/attribute tags with BNF in BG, it should rather be done in a list.)

So, an Attribute is an Event plus a value, and in the BG context that can have multiple participants (people/groups) in an event, it becomes necessary to say that an attribute applies to only one person.
The other differences (time/date) seem to have been agreed to be no reason for a difference. I think a BG event structure should be allowed to contain at least one value (actually a type value pair). If so, the only difference between an event and an attribute will be that the attribute can apply to only one person. So, if we assume that we will have lists of event types with definitions etc, as part of the standard and as a central list that can be updated, it would be possible to define in that list if an event can apply to one or several persons/group. The list could also define roles and possible types for the value.

Summed up: This way we do not need to have separate structures for attribute, we only need event. I guess this is no surprise to the rest of you.

I then started to look at various programs and it seems to me that at least three of the major ones have no distinction between event and attribute in the user interface, so we might be in good company.

As has been pointed out above, Gedcom 5.5.1 proposes an extension called "Event description" (following the EVEN tag), which does not seem to be the same as an Attribute value (which some programs seems to think). Depending on interpretation of 5.5.1 Event description may or may not be used for a person. To me this description seems to be some sort of a summary of the event, it is an event value thing not a type thing, and it seems not to be an attribute in the same sense as eg. a caste name. Since it is following the EVEN tag, it seems to apply only to user defined events. How would this event description appear in a sentence in a report?

5.5.1 talks, in EVENT_OR_FACT_CLASSIFICATION, about subtypes of ANY event type, using TYPE. The question is if this is used by anyone? Is it needed? If so, the standard and the central list can define subtypes. Separate requirement?

Adrian, the name and sex issues should perhaps be discussed separately? A name will most likely, in my mind, be a more complex thing than just a string.

About Tom's vital events. It is clear that you need to separate one Burial event from another, but that can easily be done in a level zero event. I see no problem with creating an event record even if the info comes from my grandmother. Regarding extra overhead, I am not very concerned about that, I guess a few photos transferred together with the BGfile will in many cases be much bigger than the BGfile. But, this is an encoding issue, and it will probably be affected by a solution to the evidence-conclusion issue, so I suggest that we wait with this. I agree that the BIRT event may be a special case.

Geir

Geir - "5.5.1 talks, in EVENT_OR_FACT_CLASSIFICATION, about subtypes of ANY event type, using TYPE. The question is if this is used by anyone?"

It's certainly the way that user-defined events or attributes are supposed to be written - see discussion about Custom tags on Developer's meeting page.

I already added Requirement Syntax08 "It should be possible for user-defined events, properties, characteristics, etc, of individuals, etc, to inherit features from previously defined events, properties, characteristics, etc." and speculated / suggested "If events etc are given a type and sub-type, then it would be possible for the user to create a user-defined subtype of an application defined type, and thus inherit the processing done for that type.
For instance, an event "Marriage - civil" might have a type of "Marriage" and a subtype of "civil", thus automatically doing all processing created for the event-type of "Marriage" "

So a sub-type solution is already there in a fashion.

And yes, Name will require more complexity in structuring than the usual "attribute".

As for your looking in more detail at the GEDCOM Spec'n and confounding our expectations - yes, I think we're all doing some of that! Old saying: "If you think you understand what's going on here, you obviously don't..."

PS - I shall be sorry to lose your Trabant!

... and GEDCOM is just suggesting that if something takes longer than a day, then it is *probably* a fact rather than an event. They do not impose it as a rule.

---

Adrian said Today 2:57 pm:
re: Custom GEDCOM tags

Louis - re your statements "Events also having descriptions, e.g.:
"1 EVEN Appointed Zoning Committee Chairperson
"2 TYPE Civic Appointments"
and
"events can have descriptions (i.e. attributes), the presence or absence of an attribute cannot be used to define the difference between events and facts"

That's, ahem, "interesting". I just double checked GEDCOM 5.5 and the INDIVIDUAL_EVENT_STRUCTURE in that copy seems clear to me that it does not allow a description (i.e. attribute) for an event - not even the EVEN generic event that you quote. TYPE, yes, no problem with that.

Do you know if previous (or post 5.5) versions of GEDCOM relaxed this? Or is it simply a case of software suppliers trampling over the standard again? In a sense, it doesn't really matter either way because if there are files out there with that construction, we need to deal with them. But I'd still like to understand what's going on (my pedantic brain again).

---

Adrian:

The example:

"1 EVEN Appointed Zoning Committee Chairperson
"2 TYPE Civic Appointments"

was taken right from GEDCOM 5.5.1 page 48 in the definition of EVENT_DESCRIPTOR. It says:

EVENT_DESCRIPTOR:= {Size=1:90}
Text describing a particular event pertaining to the individual or family. This event value is usually
assigned to the EVEN tag. The classification as to the difference between this specific event and other
occurrences of the EVENt tag is indicated by the use of a subordinate TYPE tag selected from the
EVENT_DETAIL structure. For example;
1 EVEN Appointed Zoning Committee Chairperson
2 TYPE Civic Appointments
2 DATE FROM JAN 1952 TO JAN 1956
2 PLAC Cove, Cache, Utah
2 AGNC Cove City Redevelopment

Now go to the FAMILY_EVENT_STRUCTURE on page 32, and you'll see under it is:

n EVEN [<EVENT_DESCRIPTOR> | <NULL>] {1:1} p.48
+1 <<FAMILY_EVENT_DETAIL>> {0:1} p.32

But, if you look at INDIVIDUAL_EVENT_STRUCTURE on page 34-35, you'll see:

n EVEN {1:1}
+1 <<INDIVIDUAL_EVENT_DETAIL>> {0:1}* p.34

which does *not* have the EVENT_DESCRIPTOR. I am sure this is what you looked at.

However, the latter MUST be a mistake in the GEDCOM definition, because their own example they gave of being "Appointed Zoning Committee Chairperson" is not a family event, but it is rather an individual event.

I believe the proper interpretation should be that it was GEDCOM's intention to have the event descriptor on both INDI events and FAM events, because it simply doesn't make sense to have them on only FAM events.

Louis

I hesitate to point this out once again, but it seems that I may be the only one stressing the point.

The discussion above covers the LEVEL ONE GEDCOM events, what I call VITAL EVENTS to separate them from the TRUE EVENTS, which stand alone as evidence and may refer to multiple persons as role players.

Since vital events are level one entities within level zero INDI records they are easily viewed as simply as a more structured kind of PFACT than the simpler PFACTs like name, age, occupation. I can see that there is interest in defining exactly what is the distinction to be made between vital events AS PFACTs and other simpler PFACTs, but I fear this entire discussion simply takes away from the important question of LEVEL ZERO TRUE EVENTS.

I really don't think it matters whether LEVEL ONE VITAL EVENTS are considered just another kind of PFACT or not. I can't think of any major reason. If you decide you can give dates to all PFACTs (e.g., occupation, name, sex [considering the possibility of sex changes -- joke, joke, joke, please]) then the differences seem to be pretty moot. Would there be any real difference in the model, the file formats, the information content, the application software, if this distinction wasn't considered to be important. Couldn't we just say that persons have lots of different kinds of PFACTs, describe their different natures, and simply fit VITAL EVENTS in as one of those sub-types?

What must distinguish Better GEDCOM is the work to extend beyond the GEDCOM model into areas that make the new model compelling for the next generation of genealogical applications. In my view we have identified the two major areas where the model must be extended, to LEVEL ZERO EVENTS as first class citizens, and to full support for persons and events at the EVIDENCE and CONCLUSION/HYPOTHESIS levels.

Anyway, I guess it's interesting to discuss the differences between LEVEL ONE VITAL EVENTS and PFACTs in general, but please don't loose sight of where the true work lines, in the definition of the LEVEL ZERO TRUE EVENTS.

Tom W.

Tom,

Might I suggest that some of this information, mostly what is in Caps, be Documented on this Wiki. YOU know what all of these means, but I am not sure that others know, specifically, what you mean. That may be why you are answering the same questions frequently.

What is: Level One Vital Events
What is: PFACTs
What is: Level Zero True Events.
What is: Evident Level
What is: Conclusion Level
What is: Hypothesis Level

Seems to me, that these definitions need to be at an Overview of the BetterGEDCOM level on this wiki. I don't know exactly where, but at a high enough level on the wiki, so that we can see what they mean.

I know, I am asking as a simple End User, but don't know how they would show up in a GEDCOM file (today). If I could, then I would be able to find it in the software that I use.

IF, they are not in a current GEDCOM file, thats OK too, but how would I see them in a BetterGEDCOM file?

Thank you,

Russ

Louis
I surrender! (grin) In fact, I surrendered sometime last night when thinking about how I had entered people's inheritances. I did like the idea of an event being a change but couldn't square it with the bit about "values" needing attributes (to use old GEDCOM terms). Coming into an inheritance was, from various viewpoints,
- an event when considering the English definition of the word event;
- an event when viewed as a change of state (in monetary value or possessions);
- an event because it involves at least 2 people (the recipient and the deceased) and therefore needs to be what Tom refers to as a Level zero event (referring to what the GEDCOM _would_ look like if it allowed the concept);
- an attribute because I needed to access the field in my application program to put the description of the inheritance in.

3-1 to events. And then Louis comes up with the text from 5.5.1 that would release that field to an event as well as an attribute. 4-0 to events.

(Louis - thanks for guiding me to the relevant pages. It's all quite different from 5.5 and I'd made the mistake of believing the section on "Modifications in Version 5.5.1", where it mentions nothing of these changes.)

So - where does that leave things?

Pretty much as Tom contends, I think.

We have an English definition of "Event = a change in state" (sorry if that's a bit like a physicist's speak). This implies a definition of Attribute (to use the old GEDCOM term) as an ongoing state-description, but everything gets a bit vague when you consider short term attributes - for instance, you could consider Occupation as an attribute but only be in a job for a day.

Fundamentally, I see no reason to worry about the distinction between an attribute for a person and an event that applies only to one person - there's no serious distinction that I can envisage in the future BG Data Model. Which I think both Louis and Tom said above - but I needed to convince myself.

The most crucial feature is what Tom has stressed above (and sorry, Tom, I was taking it as read so much that I didn't mention it) - the need to accommodate Events that have more than 1 person. How do we identify these? In Data Modelling terms, it's easy - there is a many-many relation between these Events and People (I'm talking people but it applies equally to Families / Groups / Locations etc), whereas between the other sort of events OR attributes there is only a many-to-one relation from them to People etc. But I'm not sure how to explain it in English - or whether we need to if we can communicate with the software suppliers.

I'm half tempted to say that all things known as events should be regarded as the many-to-many type of events. Even the death event, for instance, which might be thought of as an event applying to 1 person only, might include a 2nd person (in the role of a doctor, e.g.).

I'm also now half tempted to revert to a position I held some years ago and say that in BetterGEDCOM events and attributes are the same thing after all, and the only thing we need to worry about is whether they are many-to-many or many-to-one to people etc.

Adrian,

I've always used GEDCOM 5.5.1 and I also didn't realize the big change that they here from 5.5

To sum this up so that everyone see what changed, in GEDCOM 5.5 the EVENT_DESCRIPTOR on the TYPE tag, and nothing on the EVEN tag, i.e.:

1 EVEN
2 TYPE <EVENT_DESCRIPTOR>

GEDCOM 5.5.1 changed the actual meaning of the EVENT_DESCRIPTOR and moved it to the EVEN tag, and added an EVENT_OR_FACT_CLASSIFICATION to the TYPE tag, i.e:

1 EVEN <EVENT_DESCRIPTOR>
2 TYPE <EVENT_OR_FACT_CLASSIFICATION>

with the mistake I earlier pointed out of GEDCOM 5.5.1 leaving the EVENT_DESCRIPTOR off of INDI events.

But this leads us to another minor issue, which should be agreed on (maybe in another thread):

Should we mostly refer to GEDCOM 5.5 (the standard) or GEDCOM 5.5.1 (the beta) or enumerate both as the starting point of BetterGEDCOM? As we're pointing out, there were many subtle changes between the two.

Personally, I prefer GEDCOM 5.5.1, because that was the intended direction that GEDCOM was going, and many programs did adopt it. For my program I take 5.5.1 and extend it so that dropped features from previous GEDCOM versions can also be handled.

Louis

Seems like a consensus is emerging.

The following will be repeating what has already been said, but just to sum up my understanding.

We have two types of vehicle that can transfer info
- One small and simple one that has traditionally been called attributes, which are single predefined values or (user defined) type and value pares. They apply to one record (person, place, group etc) and there may me one or more of the same type.

- One big and complex one that has traditionally been called events, that should be a top level record, that can carry info that refers to zero or more person names/group names with various roles, zero or a few dates or a period, zero or more place names (possibly also identifying an address/contact structure), zero or more values (preferably predefined types), zero or more notes of various types, zero or more citation references (not determined at what levels in the event structure), zero or more multimedia references, and has administrative info (references to research process info, selection flags/marks etc.) The conclusion variant of this vehicle should have several pieces with surety info at various places in the vehicle.

There is no rule for what types of info that can be transported by these vehicles, except that types that may AT SOME (FUTURE) TIME not fit into the small one, must use the big one.

There are no rules about time (except that the small vehicle can not transfer dates– the time/period associated with the types of values in the small one is undefined or implied by the type).

The info types that can be transferred are defined in the standard, or a central registry or can be user defined, where the ONLY vehicle type to be used for an info type is specified, together with the roles, subtypes/classes and value type(s) (and possibly more). User defined types should be described in the file header for both vehicle types.

The problem seems to be what we should call them (perhaps Trabant and Rolls Royce). The names should not imply anything about what type of info they can transport.

The evidence-conclusion issue is orthogonal to this discussion, ie it should be another dimension – I hope.

Many of the lists of info types described above, carried by the big vehicle, is not central to the discussion in this topic.

---

Re. Gedcom 5.5 or 5.5.1 Agree that there is a problem, but I think, when needed, a reference should specify 5.5 or 5.5.1 – it is difficult to choose. The best thing would be to copy the text so many readers don’t have to do a lookup.