JCN 18-NOV-88 12:32 N-AUGMENT 131324

o

Functional Requirements For
Hypertext Interchange Standards

by James C. Norton
Augmentation Systems Consultant
Lasalle, Quebec

for the Graphic Communications Association
The Range and Power of Standards & the Desktop Conference
November 16-18, 1988 0

o

[>] Introduction 1
[>] Background 2
[>] Hypertext 3
[>] The AUGMENT System 4
[>] The Document Environment 5
[>] Conclusion 6
[>] References 7

o

o
o

[>] Introduction 1

I am sitting here in Lasalle, Quebec, three-button mouse in hand, keyset chords transferring my thoughts into this old PC. It is no longer the model I want, but enough to meet my needs -- and the price is right. I should link back to this thought in case I forget. 1A

Lasalle is about 3,000 miles from Menlo Park, California, where I spent most of my business life working with Doug Engelbart and his people developing what we have called an augmentation system. It is comforting to know that I am still only a few keystrokes away from him and his associates. 1B

Topic, space and time make me resist telling all, but creative rationalization permits me to bend parts of the NLS/AUGMENT story to my needs. I interpret the title words this way: 1C

Functional - what users get, not how systems are built. 1C1
Requirements - the considerations that seem to be important. 1C2
Hypertext - computer-linked information and more. 1C3
Interchange - from one system environment to others, both ways. 1C4
Standards - for compatible functionality. 1C5

[>] Background 2

Doug Engelbart. Or is there just one word for hyperthought? 2A

I met Doug in February 1958 at Stanford Research Institute in Menlo Park, where we were both starting out in new jobs. I was to be a research engineering administrative assistant; he was a young computer research engineer with internalized dreams unknown to those who hired him. 2B

By 1961, he had gained sponsorship from the Air Force Office of Scientific Research to lay out the conceptual framework for a program concerned with augmenting human intellect. He and almost 200 people who joined and later moved on along the way, worked on his concepts within that framework (Ref1). for the next 27 years. This group moved from SRI to Tymshare to McDonnell Douglas, while developing, using, and demonstrating the NLS/AUGMENT system. Our long term program came to an end in October 1988 when MDC's Augmentation Systems Division finally evaporated; but Doug's hyperthoughts are still there to be addressed. 2C

Briefly, an augmentation system has these basic components (Ref2): 2D

Doug's is not the first augmentation system and certainly not the last. Selecting computer support as a basis for his system, he went about designing a system that changed and improved many ways people can work with information. Some of his developments have been accepted and further developed by others: 2E

Some aspects are not yet fully available in the marketplace: 2F

One very useful innovation may never be accepted in the marketplace: the five finger keyset. I've long thought that the energy it takes resisting it would be better spent learning it. Things go so much faster with the keyset when you're using a mouse and keyset chord combinations. 2G

There is more to AUGMENT than the features listed above, and far more to an augmentation system than the supporting software. It is important to note that one component such as hypertext should be considered in the context of all the other aspects of such a system. A change in one component will likely affect others, causing reverberation effects through the whole system. 2H

My vantage point: I watched the program start and then evolve during its first 8 years; I then participated directly in various development and management roles for the next 20 years, always as a heavy user of the system. When I joined in 1969, I started off with a mouse, keyset, display, structure, links, viewing options -- much of what is now AUGMENT. My perspectives are based on over 20,000 hours at NLS/AUGMENT workstations and PC's and from writing about 20,000 mail messages and larger documents in an advanced hypertext environment. I recall doing more than just writing one message per hour though. It's amazing how many different types of skills are needed to live in such an environment and how much easier continual learning becomes when using such a system. 2I

Our augmentation system was intended to support a wide range of activities and to be used by individual users, groups, organizations and eventually institutions. This is still the overall context in which we think. The nature of the system required to bring significant new support to such a wide range of activities meant that a great number of innovations had to be made just to satisfy basic requirements. These included changes in hardware, software, and users' work methods. Doug's invention of the mouse is just one small example arising from such needs. Another innovation was what has come to be called hypertext which also required new ways to structure, view, and reference information in the augmenting environment. 2J

[>] Hypertext 3

Hypertext can be thought of as ranging from the basic linked screen/view level to advanced environments designed to support complex linked-information communication. 3A

I think in terms of hypertext connections whenever I work with textual information now; it's become second nature. When I think of hypertext as a concept, Doug, Ted Nelson, and also David Evans, the Australian in our group come into my mind. Evans emerges because in addition to being a delightful personality, he made the most extensive use of our structure, viewing, and linking capabilities in 1971. 3B

David wrote his Ph.D. thesis in NLS as a last step toward his degree from Stanford. He described and discussed a Dialogue Support System for planners. It was mostly about the possibilities presented by hypertext within permanently recorded (Journal, library) documents. He wrote the nearly 400 page document online, structuring it extensively, constructing an elaborate in- document link network that almost obscured the text, and printed it out for submission. He received a preliminary rejection. Submission came later. 3C

The problem was twofold. 3D

First, his professors had never read such a heavily cross-referenced document before and felt uncomfortable searching from place to place in the document trying to understand the thread of logic David was trying to present. 3E

Second, he did such a thorough and clever job of linking words, passages, sections, and chapters that he lost control of the story presentation without knowing it. He was too close to the technology at that stage and thinking more about the medium than the message -- a shared weakness. 3F

They gave him several months to rewrite and submit it, but he didn't have that much time because he was scheduled to return to Australia soon. The problem was solved in short order, as it turned out. With mouse and keyset in hand, he reassessed the real message he was trying to convey, quickly reordered chapters and sections, deleted many internal link references and some redundant text, created a few, more appropriate links and reprinted the thesis. This took a weekend. It was reread and immediately accepted. This time, he had it right, links and all. 3G

The Evans thesis was produced about a year before we started permanently recording and saving such documents in what we have called Journal libraries. His online version appears to be lost or at best somewhere on a very old, unreadable SDS-940-format tape. Assume for the moment that we now retype the final version into some other system with hypertext capabilities and import it into AUGMENT. We would be confronted with a set of problems relating to structure and links. A few questions would be: 3H

If there were standards that coordinated common structure and reference functionality to at least some reasonable level of specification, such exchanges could be quite workable. If there are not, such exchanges would lose much of the power offered in a hypertext environment and could distort the ideas being communicated by authors. With text, graphics, video, and sound now being incorporated into new hypertext systems, the need for such standards becomes even more important. To add more perspective to at least part of the issue, let's examine the text environment and hypertext functionality more closely. 3I

[>] The Augment System: a text-based example 4

[AUGMENT is not commercially available now--consider it an example.] 4A

The objectives behind our developments were: 4B

The system is incomplete, as it has almost no graphics capability and only the beginnings of organizational methodology development. Still, the AUGMENT basics and experience are worth careful consideration. Though there is much to come as new augmentation systems emerge and evolve, a lot has already been learned. 4C

A paper on the AUGMENT system would cover the following topics among others: 4D

Moving toward hypertext again, I will deal mainly with the document environment. 4E

[>] The Document Environment 5

SIMPLE TO COMPLEX documents are accommodated:
1 line messages
400 page reports
4,000 page reference sets (sections linked)
Millions of pages in distributed libraries (cross-linked) 5A

Structure (managed by internal system software) 5B

Statements are the basic structural entities in AUGMENT. These could be thought of as roughly equivalent to cards or screens in other hypertext systems. Statements can be up to 2,000 characters long and can be placed in structures up to 63 levels deep. 5C

A structurally numbered example: with statement numbers 

1 Cars
     1a Ford                   
          1a1 Passenger              
          1a2 Truck                 
     1b Chevrolet 
          1b1 Passenger
          1b2 Truck
2 Planes
     2a Boeing
     2b Lockheed
     2c McDonnell Douglas  5C1
A serially numbered example: with statement identifiers (SIDs) 
          
01 Cars                                
     02 Ford                                
          03 Passenger                 
          04 Truck                        
     05 Chevrolet                      
          06 Passenger                  
          07 Truck                            
          012 Van                      
08 Planes                            
     011 McDonnell Douglas                   
     09 Boeing                                    
 Note
     010 Lockheed (2b) was deleted  
     011 was moved after 08 and kept  its SID     
     012 Van was added after 07  5C2

Statements can have names, with displayed or suppressed viewing as with statement numbers and SIDs. First words such as Planes above can be designated as statement names. With structure, come the "nouns": up, down, successor, predecessor, origin, head, tail, end, next, and back for relative in-file addressing. 5C3

These include the statement's structural position in the file, its serial identifier, whether the statement has a name, what its name delimiters are, the "signature" (who last changed it and the date and time of the change), whether the statement contains text, graphics or other data and what portions of the statement's data can or cannot be edited. Such internal property lists are hierarchical and can be extended to include more types of data as needed. They are directly attached to each individual statement and follow them wherever they are moved or copied. 5C4

Views 5D

With structure, the user can tailor views for browsing or link referencing. Users can quickly and easily limit or expand each view as they wish. The number of levels and/or the number of lines to be displayed from each statement can be varied at will. 5D1

A wide variety of other view specifications is available and they quickly become a basic part of a user's approach to working with files. They can be intermixed and include specifications such as: 5D2

Show all branches (hierarchically tiered statements)
Show this branch only
Show all branches with the same up statement
Show only statements meeting content filter specifications
Show statement numbers (structural)
Show statement identifiers (as sequentially entered)
Show statement names
Show statement signatures (user, date, and time of last edit)
Show blank lines between statements 5D2A

Links 5E (user defined paths)

A link provides a recorded specification and path to information views. 5E1

A complex link: 5E1A 

   (office-8,norton,budget,labor.d."Benefits".+4w:wh  t i;["Vacation"];)
        1      2       3     4   
                                 5     6       7  89 10 11    12  5E1A1
A simple link: 5E1B 
    (budget,labor.d:w)
 ...   3      4   5 8 :..  5E1B1

Omitted fields imply the current  environment and settings.  5E1C1

Alternate link delimiters are angle-brackets and (See--.....)  5E1D

AUGMENT links are a part of the document in which they are placed and reflect the author's attempt to communicate a particular view or set of ideas. They are entered as viewable, editable text and stay with documents as they move through draft, final version, distribution, and recording stages. Links are permanent objects within Journal library documents. We have found much value in being able to see such links in their context, even when some are fairly complex. If desired, though, they can be placed in less obvious locations. We can also see value in approaches that hide link details completely, offering viewers only a simple icon or other indication of a link's presence. I'd prefer to have both arrangements as options. We see value in processes or services that can maintain separate databases that keep track of links between documents. But for us, that capability should be built on top of or in conjunction with document-resident links. It's almost a question of link ownership -- is it the author's or "the system's" reference? 5E2

AUGMENT links specify: location, views, and content filters 5E3

LOCATION -- assuming this network 5E3A

(1) Host computer (Server)
(2) Directory
(3) File

In-file location, by
Number (not used in above examples)
Structural statement number
Permanent statement identifier (SID)

(4) Statement name
(5) Relative structural position
(6) Data content
(7) Relative textual position
Special markers (not used in above examples) 5E3A1

VIEW SPECIFICATIONS: selected data to display, print, edit. 5E3B

(8) Structural levels [w = all lines & levels]
(9) Structural areas [h = all branches]
(10) Statement lines [t = 1 line only]
(11) Program switch (on/Off) [i = filter switch on] 5E3B1

PROGRAM control (content filters) 5E3C

(12) Within link. ["Vacation"] = Vacation anywhere in a statement 5E3C1

External to link (set to on/off as view specification) 5E3C2

Further extended links could also specify: 5E3D

External program name and input variables

Position/area within specific graphics
Color
Shading
Brightness
Size
Movement
Direction
Speed
Distance
Sound level
Sound tuning
Context classification
  • Goal
  • Requirement
  • Issue
  • Position
  • Argument
  • Decision
  • Comment
  • Request .... Etc. 5E3D1

My experience with links in AUGMENT seems to indicate that the most used and useful textual links are probably those that simply point to a particular view and lead the reader from his/her present context to that view. Given the ability to easily return to the original context and continue on, readers leave most links behind, never to return. True, for study purposes, there are times when one wants to follow a trail of many links in series, with intermediate browsing along the way perhaps, but this is done less frequently. Even so, when one does want to study or to package applications, hypertext open-ended browsing capabilities are extremely useful. 5E4

Filters 5F

Another aspect of view specification concerns the ability to filter and/or rearrange the data being referenced. In AUGMENT, users have the ability to specify simple or complex context searches and then display or print only those passages that meet the pattern specified. User programs can be utilized to specify the search criteria and also to rearrange or add to such data temporarily or permanently. 5F1

Structure: Import And Export - interfacing 5G

We have imported documents from other systems for many years. Living in a commercial and government computer network environment, there are thousands of messages and documents that flow in and out of every system. It is easy to bring ASCII files into AUGMENT on a line by line basis, but getting such files into hierarchically structured form has required some specialized software. Users are provided with several basic options that cover most cases. Users also have the option of using any special content analysis/reformatting programs they may provide. 5G1

The basic options are: 5G2

Special content analysis programs 5G3

Our importing process still has some problems 5G4

Journal Library: Future Access and Fixed References 5H

Links aren't worth much if they point at something that no longer exists or that has been moved with no forwarding address. Therefore, AUGMENT users have come to rely on an online Journal/Mail library system for other than most draft or personal information. This system optionally records any mail message or document file permanently, assigning a unique, citable Journal document number and preserving in-file addresses. 5H1

Once a document has been entered into the Journal system, it cannot be changed. This permits reliable future reference to such documents or passages within them -- right down to any character position. 5H2

System software keeps track of the actual location of such documents in each AUGMENT server on a network. An archiving system manages the physical location of such files and provides automatic notification to operations people when files that may no longer be online are requested. 5H3

Cataloging and searching capabilities are available that facilitate browsing and analysis. These make use of links also. 5H4

Without such a document management system, link networks are likely to fall into serious disrepair with subsequent loss in user confidence and use. 5H5

Link-Based Applications 5I

Given all of the above, the opportunity to build applications on the hypertext information network becomes significant. Computers can follow links even more easily and faster than users. Task management systems, budget planning packages, query systems, reference set guides, presentation kits, help systems, and so on can work with more data more effectively in such an environment. 5I1

At SRI, Tymshare, and McDonnell Douglas, my associates and I were involved with the development and implementation of many applications, both in experimental and operational settings. These involved many hundreds of users and focused our system on significant organizational information problems. 5I2

As an example, the "Suspense System" we developed for the Air Force is intended to support a wide range of user and organizational needs relating to task and project management. Goals, requirements, plans, task assignments, remote collaboration, technical data, document production, reports, reviews, approvals, and final disposition of hundreds of mission-specific operations must be coordinated. Equipment and software, documentation, user training, and continual adaptation to newly surfaced needs were central to the endeavor. 5I3

My point is this: an augmentation system to support such activities relies heavily on links and Journal library document communication and management in the hypertext environment. With the ability to create, manipulate, reference, retrieve, integrate, and use hypertext data between different systems and networks, we are approaching Engelbart's next augmentation system stage: really putting it to work at the organizational level on a broad scale. From what I've seen, that's probably the hardest part. 5I4

[>] Conclusion 6

There are many addressing and view specification needs that must be met to provide users with the ability to link and effectively portray views of textual information in hypertext environments. Add graphics, video, and sound and those needs increase significantly. With hypertext system developers working independently with different design goals on one hand, and distributed users exchanging more and more information between systems on the other, the need for hypertext interchange standards becomes more critical. 6A

Part of the problem in developing such standards will be in determining how to deal with the interchange of information between systems having different levels of capability. This will involve decisions that go beyond simple translation mapping of protocols and treatment of data that cannot be handled by some systems. Decisions will have to be made about the treatment of non- exact interchange and the impact on both authors and readers in other systems. 6B

While intra-file linking is very useful, the inter-file linking between text passages, graphics, and other data is where the power of hypertext waits. In order to maintain and control access and reliability in hypertext environments, document management systems with at least the capabilities of the AUGMENT Journal system will be essential. 6C

[>] References 7

Ref1. "Augmenting Human Intellect: A Conceptual Framework," Summary Report, Stanford Research Institute, Summary Report AFOSR-3223 under Contract AF 49(638)-1024, SRI Project 3578 for Air Force Office of Scientific Research, October 1962, pp. 1-139. Engelbart, D. C. (Augment,133182,) 7A

Ref2. "Evolving the Organization of the Future: A Point of View," Proceedings of the Stanford International Symposium on Office Automation, March 1980, 6 pp, Engelbart, D. C. (Augment,80360,) 7B

Extended bibliography 7C

Note -- About the purple numbers on some documents .... 7C1

Beach

Last Edited: 16 February 97