Memento: Updated Technical Details (March 2010)

Memento: Time Travel for the Web 

The Memento Team

Herbert Van de Sompel
Michael L. Nelson
Robert Sanderson
Lyudmila Balakireva
Scott Ainsworth
Harihar Shankar

Memento is partially funded by the
Library of Congress

 
Updated Technical Details (March 2010)
Memento: Time Travel for the Web
Updated Technical Details (03/2010)

Memento wants to make navigating the Web’s Past Easy

http://www.mementoweb.org
http://groups.google.com/group/memento-dev
2

Recap of the Basics …

3

W3C Web Architecture: Resource – URI - Representation

dereference

URI

Identifies

Resource

Represents

Representation

4

W3C Web Architecture: Resource – URI - Representation

dereference content negotiation

URI

Identifies

Resource

Represents

Representation 1

Represents Representation 2

5

Problem Statement …

6

Resources

7

Resources have Representations

8

Resources have Representations that Change over Time

9

Only the Current Representation is Available from a Resource

10

Old Representations are Lost Forever

11

Archived Resources Exist

12

Sep 11 2001, 20:36:10 UTC Dec 20 2001, 4:51:00 UTC

Archived Resources

http://en.wikipedia.org/w/index.php?
http://web.archive.org/web/20010911203610/http:// title=September_11_attacks&oldid=282333 archived
www.cnn.com/ archived resource for http://cnn.com resource for http://en.wikipedia.org/wiki/
September_11_attacks

Updated Technical Details (03/2010) 13

Finding Archived Resources

Go to http://www.archive.org/ and search On http://web.archive.org/web/*/http://cnn.com, select
http://cnn.com desired datetime

14

Finding Archived Resources

Go to
http://en.wikipedia.org/wiki/September_11_attacks Browse History
and click History

15

Dec 20 2001, 4:51:00 UTC current

Navigating Archived Resources

Pentagon

title=September_11_attacks&oldid=282333 archived
http://en.wikipedia.org/wiki/The_Pentagon
resource for http://en.wikipedia.org/wiki/
September_11_attacks3

16

Sep 11 2001, 20:36:10 UTC Sep 11 2001, 21:38:55 UTC

Navigating Archived Resources

SPACE

http://web.archive.org/web/20010911203610/http:// http://web.archive.org/web/20010911213855/
www.cnn.com/ archived resource for http://cnn.com www.cnn.com/TECH/space/

17

Current and Past Web are Not Integrated

•  Current and Past Web
based on same technology.

•  But, going from Current to
Past Web is a matter of
(manual) discovery.

•  Memento wants to make
going from Current to Past
Web a (HTTP) protocol
matter.

•  Memento wants to integrate
Current And Past Web.

18

The Memento Approach …

19

Navigate the Web of the Past

http://en.wikipedea.org/wiki/
Web_Archiving

20


Web_Archiving
Oct 11 2009, 05:30:33 UTC

Set browser time dial to …

21


http://en.wikipedea.org/wiki/ From Wikipedia History
Web_Archiving
Oct 11 2009, 05:30:33 UTC Oct 01 2009, 16:30:00 UTC


22


Web_Archiving
Oct 11 2009, 05:30:33 UTC Oct 01 2009, 16:30:00 UTC


Robots Exclusion Protocol Oct 11 2009, 05:30:33 UTC

23


Robots_exclusion_protocol
Oct 11 2009, 05:30:33 UTC

Browser time dial still at …

24


Oct 11 2009, 05:30:33 UTC Sep 15 2009, 20:49:00 UTC


25


Oct 11 2009, 05:30:33 UTC Sep 15 2009, 20:49:00 UTC


Robots Exclusion Oct 11 2009, 05:30:33 UTC

26


http://www.robotstxt.org/

Oct 11 2009, 05:30:33 UTC


27


http://www.robotstxt.org/ From Internet Archive

Oct 11 2009, 05:30:33 UTC Nov 09 2007, 06:21:04 UTC


28

How does Memento achieve this?

There are two components to the Memento Solution:

•  Component 1: Navigation towards an archived
resource via its original resource, by leveraging
content negotiation.

•  Component 2: A discovery API for archives that
allows requesting a list of all archived versions it
holds for a resource with a given URI.

29



30

The Web without a Time Dimension

31


32


Need to use a different URI to access archived versions of a resource and its current version

33

The Web with Time Dimension added by Memento

In Memento: use URI of the current version to access archived versions, but qualify it with datetime

34


… and magically arrive at an archived version

35


In order to fully understand how Memento introduces
a time dimension to the Web, we present a brief
recap of Transparent Content Negotiation (conneg)
in HTTP.

RFC 2295. Transparent Content Negotiation in HTTP,
http://www.ietf.org/rfc/rfc2295.txt

36

HTTP GET on URI A

37

GET with conneg on URI T – Server Choice – 302 Found – Step 1

transparently
negotiable
resource

38

GET with conneg on URI T – Server Choice – 302 Found – Step 2

39

GET with conneg on URI T – Server List – 406 Not Acceptable

40

How does Memento do This?


41

Terminology Intermission

We introduce the term Memento to refer to an
archived version of a resource.

A Memento for a resource URI-R (as it existed)
at time ti is a resource URI-Mi [URI-R@ti] for
which the representation at any moment
past its creation time tc is the same as the
representation that was available from URI-
R at time ti, with tc >= ti. Implicit in this
definition is the notion that, once created, a
Memento always keeps the same
representation.

42

DT-conneg: Content Negotiation in the datetime dimension

•  RFC 2295 introduces conneg in the following dimensions: media type,
language, compression, character set, e.g.:
-  HTTP Request:
o  Accept-Language: en-US

o  HTTP Response:

o  Content-Language: en-US

•  Inspired by RFC 2295, Memento introduces datetime conneg:
-  HTTP Request:
o  Accept-Datetime: Mon, 12 Oct 2009 14:20:33

GMT
o  HTTP Response:
o  Content-Datetime: Sun, 11 Oct 2009 11:18:05
GMT
43

DT-conneg: Content Negotiation in the datetime dimension

•  This means that somewhere, we will need transparently negotiable
resources (cf. slides 38-40) that supports the datetime dimension to
get to appropriate Mementos.

•  This will be discussed for 2 classes of servers:
o  Web servers without internal archival capabilities;

o  Web servers with internal archival capabilities.

44

Servers Without Internal Archival Capabilities

•  This type includes:
o  Servers that are crawled by a web archive, such as the

Internet Archive
o  Servers with an associated transactional archive

•  These servers are not aware of the details of Mementos of their
resources held by external archives.
•  These servers do not have the essential information (URI-Ms,
and associated datetimes) to respond to a DT-conneg request.

•  But they can be constructive by pointing (HTTP Link) a client to
an archive that can respond to the DT-conneg request.
o  Unconditionally do this for resources for which Mementos are

conceivably available in the archive.
45

Oct 04 2009, 12:00:01 UTC

current

Oct 10 2009, 12:00:03 UTC

http://lanlsource.lanl.gov/
hello

Oct 21 2009, 12:00:01 UTC
http://mementoarchive.lanl.gov/store/ta/
20091021120001/http://lanlsource.lanl.gov/hello

46

original
resource Mementos

original server archival server

47

DT-conneg with URI-G to get URI-M

original
resource Mementos

transparently variant
negotiable resources
resource


48

HTTP
DT-conneg with URI-G to get URI-M
Link

original
resource Mementos

resource


49


We introduce the term TimeGate to refer to a
transparently negotiable resource that supports the
datetime dimension.

A TimeGate for an original resource URI-R is a
transparently negotiable resource URI-
G[URI-R] for which all variant resources are
Mementos URI-Mi[URI-R@ti] of the resource
URI-R. Since multiple archives may host
versions of URI-R, multiple TimeGates may
exist for any given resource, i.e. one per
archive.

50

HTTP
Link DT-conneg with URI-G to get URI-M

original
resource TimeGate Mementos

resource


51

How to redirect from Original Resource to its (external) TimeGate

•  Q1: Which archive to HTTP Link to?

o  The archive with the best coverage for the server at hand.
o  Always redirect to an Aggregator (see slides 110-139)
o  No redirection by server: client takes control, accessing its
preferred TimeGate, bypassing Original Resource.

•  Q2: What is the TimeGate URI-G for URI-R on the chosen
archive?

o  Convention for syntax of URI-G as function of URI-R.
-  http://web.archive.org/web/timegate/http://cnn.com
o  Always redirect to an Aggregator (see slides 110-139)

52

Servers With Internal Archival Capabilities

•  This type includes:
o  Content Management Systems

o  Version Control Systems

o  Servers that archive resource representations in the cloud

and keep track of the URIs and datetimes of remotely
archived resources.

•  These servers have all the essential information (URI-Ms, and
associated datetimes) to respond to a DT-conneg request.

•  The previous architectural solution is maintained to enforce strict
distinction between handling requests for current and past
representations.

53

Dec 20 2001, 4:51:00 UTC

Dec 31 2004, 20:46:00 UTC
current

http://en.wikipedia.org/wiki/
September_11_attacks

Dec 20 2008, 22:21:00 UTC

title=September_11_attacks&oldid=259237305

54

original
Mementos
resource

original server

55

HTTP
Link DT-conneg with URI-G to get URI-M

original TimeGate Mementos
resource
transparently
negotiable
resource

variant
resources

original server

56

How to redirect from Original Resource to its (internal) TimeGate

•  Q1: Which archive to HTTP Link to?

o  No problem as the archive and the original server coincide:
Original Resources, TimeGates, Mementos are on same server.

•  Q2: What is the TimeGate URI-G for URI-R on the chosen archive?

o  Can be internal convention for syntax of URI-G as function of
URI-R.
-  For example, MediaWiki:
http://a.wiki.org/wiki/Special:TimeGate/http://a.wiki.org/wiki/the_title

57

HTTP Headers in Memento

58

The HTTP Headers used in Memento …

•  Define two new headers:
–  request: Accept-Datetime:
–  response: Content-Datetime:

•  Introduce new content for two existing headers:
–  response: Vary: ; Link:

•  Use one existing headers without modification:
–  response: Location:, TCN:

59

HTTP Request Headers for DT-conneg

•  Accept-Datetime:
o  Issued against TimeGate, (Original Resource), (Memento)

o  Header content: desired datetime of Memento

Accept-Datetime: Mon, 12 Oct 2009 14:20:33 GMT
60

HTTP Response Headers for DT-conneg

•  Content-Datetime:
o  Returned by Mementos

-  Even when not as a result of DT-conneg.
o  Symmetrical with regular conneg e.g. Content-Type, Content-

Language, …
o  Header content: archival datetime of the Memento that is being

returned.

•  Note: This header is crucial to allow a client to understand it has
arrived at a Memento.

Content-Datetime: Mon, 12 Oct 2009 14:20:33 GMT
61


•  TCN:
o  Returned by TimeGate

o  Same use as in regular conneg

o  Header content:

-  Choice:
–  302 response
–  Chosen Memento in Location header
–  Alternative Mementos listed in Link header
-  List:
–  406, 300 response
–  Possible Mementos listed in Link header (and
optionally in body as HTML)

TCN: choice
62


•  Vary:

o  Similar to regular conneg

o  Header content:

-  negotiate, accept-datetime

•  Note: accept-datetime content in Vary header is crucial to
allow a client to understand it has arrived at a TimeGate.

Vary: negotiate, accept-datetime
63


•  Location:

o  Similar to regular conneg

o  Header content: Location of Memento

Location: http://web.archive.org/web/20010911223004/
http://cnn.com
64


•  Link:
o  Returned by Original Resource, TimeGate and Mementos

65


•  Link header content for Original Resource:
o  Recommended: URI-G of TimeGate

-  rel="timegate"

Link: <http://web.archive.org/web/timegate/http://
cnn.com/>; rel="timegate"
66


•  Link header content for TimeGate, Mementos:
o  Required: URI-R of Original Resource

-  rel="original" . Note: even when not as part of DT-conneg.
o  Required: URI-M of first and last available Mementos;

-  rel="first-memento"; datetime="…"
-  rel="last-memento"; datetime="…"
o  Recommended: URI-M of time-adjacent Mementos;

-  rel="prev-memento"; datetime="…"
-  rel="next-memento"; datetime="…"
o  Recommended: URI-B of TimeBundle

-  rel="timebundle"
o  Optional: URI-M of other Mementos

-  rel="memento"; datetime="…"

67


•  Link header content for TimeGate, Mementos:

Link: <http://cnn.com/>; rel="original",
<http://web.archive.org/web/timebundle/http://cnn.com/>;
rel="timebundle",
<http://web.archive.org/web/20000915112826/http://
www.cnn.com>; rel="first-memento"; datetime="Tue, 15 Sep 2000
11:28:26 GMT",
www.cnn.com>; rel="last-memento"; datetime="Tue, 08 Jul 2008
09:34:33 GMT",
www.cnn.com>; rel="prev-memento"; datetime="Tue, 11 Sep 2001
20:30:51 GMT",
www.cnn.com>; rel="next-memento"; datetime="Tue, 11 Sep 2001
20:47:33 GMT"

68

Two Memento HTTP Navigations

Details at http://www.mementoweb.org/guide/http/

69

Memento HTTP Flow

HEAD R, Accept-Datetime

LinkG

GET G, Accept-Datetime

302M, Vary, TCN, LinkR,B,M

GET M, Accept-Datetime

200, Content-Datetime, LinkR,B,M

Memento HTTP Flow: Notes (1)


•  Accept-Datetime header only essential when communication with
URI-G. Not necessary when communicating with URI-R, URI-M.

•  GET G can be optimized to HEAD G

71


LinkG

•  Link header pointing at URI-G only essential when server of URI-R has
its special-purpose of preferred archive(s) for its own resources.

•  If Link is missing, HTTP client can resort to its preferred TimeGates.

72


Three aspects of the Memento HTTP flow ensure maximal leverage of
established Web caching infrastructure:

•  URI-R separate from URI-G: Coinciding URI-R and URI-G would cause
problems because caching regimes for regular resources and negotiable
resources are different.

•  Link header pointing from URI-R to URI-G (as opposed to HTTP 302
used in prior Memento design) allows for caching of responses from URI-R
(many thanks to Erik Hetzner of California Digital Library).

•  URI-G separate from URI-M: Allows for caching of Mementos.

73


Scenario 1

•  cnn.com includes Link to TimeGate at Internet Archive
•  URI-R on one server, URI-G & URI-M on another

74

Memento HTTP Flow: URI-R


HEAD http://cnn.com/ HTTP/1.1
Host: cnn.com
Accept-Datetime: Tue, 11 Sep 2001 20:35:00 GMT
Connection: close

76

Memento HTTP Flow: Success – URI-R

LinkG

HTTP/1.1 200 OK
Date: Thu, 21 Jan 2010 00:02:12 GMT
Server: Apache
Link: <http://web.archive.org/web/timegate/http://cnn.com>; rel="timegate"
Content-Length: 255
Connection: close
Content-Type: text/html; charset=iso-8859-1

78

Memento HTTP Flow: URI-G


GET http://web.archive.org/web/timegate/http://cnn.com HTTP/1.1
Host: web.archive.org
Connection: close

80

Memento HTTP Flow: Success – URI-G

302M, Vary, LinkR,B,M

HTTP/1.1 302 Found
Date: Thu, 21 Jan 2010 00:06:50 GMT
Server: Apache
TCN: choice
Location: http://web.archive.org/web/20010911203610/http://www.cnn.com
<http://web.archive.org/web/timebundle/http://cnn.com/>; rel="timebundle",
<http://web.archive.org/web/20000915112826/http://www.cnn.com>;
rel="first- memento"; datetime="Tue, 15 Sep 2000 11:28:26 GMT",
rel="last-memento"; datetime="Tue, 08 Jul 2008 09:34:33 GMT",
rel="prev-memento"; datetime="Tue, 11 Sep 2001 20:30:51 GMT",
rel="next-memento"; datetime="Tue, 11 Sep 2001 20:47:33 GMT"
Content-Length: 0
Connection: close
Content-Type: text/plain; charset=UTF-8

82

Memento HTTP Flow: URI-M


GET http://web.archive.org/web/20010911203610/http://www.cnn.com HTTP/1.1
Connection: close

84

Memento HTTP Flow: Success – URI-M


HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
X-Archive-Orig-Accept-Ranges: bytes
…
Content-Type: text/html;charset=utf-8
Content-Length: 23364
Date: Thu, 21 Jan 2010 00:09:40 GMT
Content-Datetime: Tue, 11 Sep 2001 20:36:10 GMT
rel="first-memento"; datetime="Tue, 15 Sep 2000 11:28:26 GMT",
Connection: close


Scenario 2

•  wikipedia.org natively supports Memento
•  URI-R, URI-G & URI-M on one server

87



HEAD /wiki/DJ_Shadow HTTP/1.1
Host: en.wikipedia.org
Accept-Datetime: Thu, 05 Nov 2009 00:00:00 GMT
Connection: close

89


LinkG

HTTP/1.1 200 OK
Date: Thu, 21 Jan 2010 00:02:12 GMT
Server: Apache
Link:
<http://en.wikipedia.org/Special:TimeGate/http://en.wikipedia.org/wiki/DJ_Shadow>;
rel="timegate"
Connection: close

91



GET /Special:TimeGate/http://en.wikipedia.org/wiki/DJ_Shadow HTTP/1.1
Connection: close

93



HTTP/1.1 302 Found
Date: Thu, 21 Jan 2010 00:06:50 GMT
Server: Apache
TCN: choice
Location: http://en.wikipedia.org/w/index.php?title=DJ_Shadow&oldid=324178040
Link: <http://en.wikipedia.org/Special:TimeBundle/http://en.wikipedia.org/wiki/
DJ_Shadow>; rel="timebundle",
<http://en.wikipedia.org/wiki/DJ_Shadow>; rel="original",
<http://en.wikipedia.org/w/index.php?title=DJ_Shadow&oldid=1493688>;
rel="first-memento"; datetime="Sun, 28 Sep 2003 01:42:00 GMT",
rel="last-memento"; datetime="Tue, 12 Jan 2010 19:55:00 GMT",
rel="prev-memento"; datetime="Wed, 28 Oct 2009 14:307:00 GMT",
<http://en.wikipedia.org/w/index.php?title=DJ_Shadow&oldid=326164283> ;
rel="next-memento"; datetime="Thu, 26 Nov 2009 23:50:00 GMT"
Content-Length: 0
Connection: close
95



GET /w/index.php?title=DJ_Shadow&oldid=324178040 HTTP/1.1
Connection: close

97



HTTP/1.1 200 OK
Server: Apache
Content-Type: text/html; charset=utf-8
Date: Thu, 21 Jan 2010 00:09:40 GMT
Content-Datetime: Thu, 05 Nov 2009 23:41:00 GMT
rel="prev-memento"; datetime="Wed, 28 Oct 2009 14:307:00 GMT",
rel="next-memento"; datetime="Thu, 26 Nov 2009 23:50:00 GMT"
Connection: close

99

Memento HTTP Navigations involving codes other than 200, 302

100

300 Multiple Choices
•  Two scenarios that generate a 300 at the TimeGate:
–  A client requests a 300 using the "Negotiate: 1.0" request header
–  An archive has two or more Mementos with the same Datetime (HTTP
only supports second-level granularity)
HTTP/1.1 300 Multiple Choices
Server: Apache
Content-Length: 705
Date: Thu, 21 Jan 2010 00:09:40 GMT
TCN: list
rel=”memento"; datetime="Sun, 31 May 2009 15:43:00 GMT",
rel=”memento"; datetime="Sun, 31 May 2009 15:43:00 GMT"
Connection: close 101

406 Not Acceptable
•  A client request for a Memento with a datetime outside the first and
last values will generate a 406
•  For example a request in Wikipedia with:
Accept-DateTime: Mon, 31 May 1999 00:00:00 GMT

HTTP/1.1 406 Not Acceptable
Server: Apache
Content-Length: 709
Date: Thu, 21 Jan 2010 00:09:40 GMT
TCN: list
Connection: close 102


103



Done


104



105

Why an API?

•  Mementos for any given
URI-R are distributed
across archives.

•  In order to get a correct
perspective of available
Mementos, different
archives need to be
consulted.

•  Can do so in distributed
consultation mode
(slooow), or by
consulting an
aggregator.


We introduce the term TimeBundle to refer to a
resource via which an overview of all Mementos for
an original resource URI-R is available.

A TimeBundle for a resource URI-R, is a
resource URI-B[URI-R] that is an
aggregation of:

(a)  All Mementos URI-Mi [URI-R@ti] available
from an archive,
(b)  The archive's TimeGate URI-G for URI-R,
(c)  The original resource URI-R itself.

107

URI-
M1

URI-
URI-R M2

URI-
M3

Original
Memento
resource

108

HTTP LINK HEADER
timegate

URI-
M1

DT-conneg URI-
URI-R URI-G M2

URI-
M3

Original
TimeGate Memento
resource

Memento DT-conneg component

109

See OAI-ORE: http://www.openarchives.org/ore/1.0/toc/

ore:aggregates
URI-
M1

ore:aggregates

URI-
URI-R URI-G M2 URI-B
ore:aggregates

URI-
ore:aggregates M3

ore:aggregates

Original
TimeGate Memento TimeBundle
resource

Memento DT-conneg component

110

HTTP LINK HEADER
timebundle
ore:aggregates
TimeMap
URI-
M1

ore:aggregates
HTTP
URI- 303
URI-R URI-G M2 URI-B URI-T
ore:aggregates

URI-
ore:aggregates M3

ore:aggregates

Original
TimeGate Memento TimeBundle TimeMap
resource

Memento DT-conneg component Memento discovery component

111

The Aggregator: A Service using the TimeBundle API

112

TimeBundle API: For Discovery, Cross-Archive Services

•  Archive uses common approaches to make TimeBundles/
TimeMaps discoverable:
o  SiteMaps,

o  Atom Feeds,

o  OAI-PMH.

•  Aggregator harvests and merges TimeMaps. Based on this
information, the Aggregator exposes its own TimeGates.
o  Cross-archive

o  Finer datetime granularity

o  Better chances of matching a client’s datetime preference.

o  Can become a shared target for redirection for many web

servers.

113

Aggregatorusing TimeBundle API
Aggregator Conceptualization

Accept-Datetime

Original TimeMap
HTTP LINK
rel="timegate" server

TimeMap

Aggregator
datetime
browser content TimeMap
negotiation

HTTP LINK rel="original"
HTTP Link rel="timegate"
Content-Datetime
Memento TimeMap
server servers

114

Two Memento HTTP Navigations involving an Aggregator

115

A Memento HTTP Navigation involving an Aggregator

Scenario 3

•  www.digitalpreservation.gov points at TimeGate provided by
an Aggregator
•  URI-R, URI-G, URI-M on different servers

116



HEAD / HTTP/1.1
Host: www.digitalpreservation.gov
Accept-Datetime: Sat, 10 Oct 2009 00:00:00 GMT
Connection: close

118


LinkG

HTTP/1.1 200 OK
Date: Thu, 21 Jan 2010 00:02:12 GMT
Server: Apache
Link:
<http://mementoproxy.lanl.gov/aggr/timegate/http://www.digitalpreservation.gov/> ;
rel="timegate"
Content-Length: 255
Connection: close

120



GET /aggr/timegate/http://www.digitalpreservation.gov/ HTTP/1.1
Host: mementoproxy.lanl.gov
Connection: close

122



HTTP/1.1 302 Found
Date: Thu, 21 Jan 2010 00:06:50 GMT
Server: Apache
TCN: choice
Location: http://wayback.archive-it.org/1610/20090928171405/http://
www.digitalpreservation.gov/
Link: <http://www.digitalpreservation.gov/>; rel="original",
<http://mementoproxy.lanl.gov/aggr/timebundle/http://www.digitalpreservation.gov/>;
rel="timebundle",
<http://wayback.archive
-it.org/256/20051108162921/http://www.digitalpreservation.gov/>;
rel="first-memento"; datetime="Tue, 08 Nov 2005 00:00:00 GMT",
<http://webcitation.org/query?id=1257028234035091>;
rel="next-memento"; datetime="Sat, 31 Oct 2009 18:30:35 GMT",
<http://webcitation.org/query?id=1213058061345794>;
rel="prev-memento"; datetime="Mon, 09 Jun 2008 20:34:23 GMT",
rel="last-memento"; datetime="Wed, 20 Jan 2010 10:20:00 GMT"
Content-Length: 0
Connection: close



GET /1610/20090928171405/http://www.digitalpreservation.gov/ HTTP/1.1
Host: wayback.archive-it.org
Connection: close

126



HTTP/1.1 200 OK
Server: Apache-Coyote/1.1 Link header values are local
… to wayback.archive-it.org
Content-Type: text/html;charset=utf-8 and different than those
Content-Length: 23364 provided by URI-G
Date: Thu, 21 Jan 2010 00:09:40 GMT
Content-Datetime: Mon, 28 Sep 2009 17:14:05 GMT
Link: <http://www.digitalpreservation.gov/>; rel="original",
<http://wayback.archive-it.org/web/timebundle/http://www.digitalpreservation.gov/>;
rel="timebundle",
rel="first-memento"; datetime="Tue, 08 Nov 2005 00:00:00 GMT",
rel="last-memento"; datetime="Wed, 20 Jan 2010 10:20:00 GMT"
Connection: close

128

A Memento HTTP Navigation involving an Aggregator

Scenario 4

•  cnn.com does not include a Link to a TimeGate
•  client takes control by directly contacting its preferred
Aggregator
•  URI-R, URI-G, URI-M on different servers

129

Memento HTTP Flow


200







HEAD http://cnn.com/ HTTP/1.1
Host: cnn.com
Connection: close

131

Memento HTTP Flow: No Success – URI-R

200

HTTP/1.1 200 OK
Date: Thu, 21 Jan 2010 00:02:12 GMT
Server: Apache
Content-Length: 255
Connection: close

133



GET /aggr/timegate/http://cnn.com/ HTTP/1.1
Host: mementoproxy.lanl.gov
Connection: close

135



HTTP/1.1 302 Found
Date: Thu, 21 Jan 2010 00:06:50 GMT
Server: Apache
TCN: choice
Location: http://web.archive.org/web/20010911203610/http://www.cnn.com
<http://mementoproxy.lanl.gov/aggr/timebundle/http://cnn.com/>; rel="timebundle",
<http://archive-it.org/2245/20100310102000/http://www.cnn.com>;
rel="last-memento"; datetime="Wed, 10 Mar 2010 10:20:00 GMT",
Content-Length: 0
Connection: close

137



GET http://web.archive.org/web/20010911203610/http://www.cnn.com HTTP/1.1
Connection: close

139



HTTP/1.1 200 OK
Server: Apache-Coyote/1.1 note: Link header values are
… local to web.archive.org
Content-Type: text/html;charset=utf-8 and different than those
Content-Length: 23364 provided by URI-G
Date: Thu, 21 Jan 2010 00:09:40 GMT
Content-Datetime: Tue, 11 Sep 2001 20:36:10 GMT
Connection: close

141

The Memento Profile of OAI-ORE

142

Memento Profile of OAI-ORE 

URI-G TimeGate ORE Aggregated Resource

ore:aggregates
URI-R
Original ORE Aggregated Resource
ore:aggregates resource
ore:describes
ore:aggregates
URI-
URI-T URI-B M1 Memento ORE Aggregated Resource

ore:aggregates

URI-
TimeMap TimeBundle M2 Memento ORE Aggregated Resource

ORE Resource Map ORE Aggregation

143

Information in the TimeMap about the TimeGate 

•  Coverage Period the TimeGate has Mementos for
Predicate: mem:covers
Object: mem:TimeSpan

•  Which Original Resource it is a TimeGate for
Predicate: mem:timeGateFor
Object: mem:OriginalResource

•  TimeSpan Class:
•  mem:start datestamp
•  mem:end datestamp

144

Information in the TimeMap about the TimeGate 

mem:covers
URI-G

mem:start
mem:timeGateFor datetime
ore:aggregates
URI-R

ore:aggregates
ore:describes datetime
mem:end
URI-
URI-T URI-B M1

ore:aggregates

145

Information in the TimeMap about Mementos 

•  The representation's mime-type:
dc:format
•  The representation's byte size:
dc:extent
•  Over what period was the representation observed as the current
representation of the Original Resource::
Predicate: mem:observedOver
•  Or over what period was the representation known to be active as the
current representation of the Original Resource:
Predicate: mem:validOver
•  The number of observations of the representation in the given time period:
mem:observations integer
•  Any further information that is available about Mementos

146

Information in the TimeMap about Mementos 

integer
URI-G mem:observations

mem:start datetime
ore:aggregates
URI-R

ore:aggregates
mem:
ore:describes datetime
mementoFor mem:end

URI-
URI-T URI-B M1
mem:
ore:aggregates observedOver
integer
dc:extent

dc:format
string

147

Memento Ontology: Classes 

•  TimeMap SubClassOf ore:ResourceMap

•  TimeBundle SubClassOf ore:Aggregation

•  OriginalResource SubClassOf gen:TimeGenericResource

•  TimeGate SubClassOf irw:WebResource

•  Memento SubClassOf gen:TimeSpecificResource

•  TimeSpan Typed Blank Node

148

Memento Ontology: Relationships 

•  timeGateFor subject: TimeGate object: OriginalResource

•  hasTimeGate subject: OriginalResource object: TimeGate

•  mementoFor subject: Memento object: OriginalResource

•  covers subject: TimeGate object: TimeSpan

•  observedOver subject: Memento object: TimeSpan

•  validOver subject: Memento object: TimeSpan

149

Memento Ontology: Properties 

•  start subject: TimeSpan type: datestamp

•  end subject: TimeSpan type: datestamp

•  observations subject: TimeSpan type: integer

150

Ontologies 

Memento - mem: http://www.mementoweb.org/terms/tb/

Generic/Specific Resource – gen: http://www.w3.org/2006/gen/ont

Identity of Resource on the Web – irw:
http://ontologydesignpatterns.org/ont/web/irw.owl

Object Reuse & Exchange – ore: http://www.openarchives.org/ore/terms/

Dublin Core – dc: http://purl.org/dc/elements/1.1/

151

Sample TimeMap in RDF/XML 
<rdf:RDF>
<mem:TimeMap
rdf:about="http://mementoproxy.lanl.gov/ia/timemap/rdf/http://www.archive.org/">
<ore:describes>
<mem:TimeBundle
rdf:about="http://mementoproxy.lanl.gov/ia/timebundle/http://www.archive.org/">
<ore:aggregates>
<mem:OriginalResource rdf:about="http://www.archive.org/"/>
</ore:aggregates>
<ore:aggregates>
<mem:TimeGate
rdf:about="http://mementoproxy.lanl.gov/ia/timegate/http://www.archive.org/">
<mem:timeGateFor rdf:resource="http://www.archive.org/"/>
<mem:covers>
<mem:TimeSpan>
<mem:start>1999-10-11T06:44:03 GMT</mem:start>
<mem:end>2008-07-19T08:52:28 GMT</mem:end>
</mem:TimeSpan>
</mem:covers>
</mem:TimeGate>
</ore:aggregates>
<ore:aggregates
rdf:resource="http://web.archive.org/web/19971011064403/http://www.archive.org/"/>
…
152



Done

Done

153

Memento in a non-Memento Web …

154

Memento and Client Development

•  Browser plug-in must use different HTTP request/response cycle
when in time travel mode.

•  Browser plug-in needs to be more or less aware of its state in the
chain Original Resource => TimeGate => Memento

•  Interesting user interface challenges, among others related to
time-distribution of Mementos embedded in encompassing
Memento, e.g. images in HTML page.

155

Memento and Web Archives

•  Web Archives rewrite URLs in archived pages, in order to avoid:

o  Serving current representations of embedded resources;
o  Linking to current representations of resources

•  The upside: Archived pages are self-contained.

•  The downside:
o  Cannot navigate beyond the archive’s content;

o  Other servers (archives or original) may have appropriate

version of (missing) embedded or linked resource.

•  Memento does not require URL-rewriting.

156

BBC home page from Internet Archive; URL rewriting

157

Embedded & linked resources still available at BBC site

158

BBC home page from Internet Archive; no URL-rewriting

159

Standardization

•  Link relationships original, timegate, timebundle,
memento, first-memento, last-memento, prev-memento,
next-memento will be registered as per upcoming HTTP Link
header RFC.

•  Headers Accept-Datetime, Content-Datetime will be registered
in the provisional registry for Message Header Fields as per RFC
3864.

•  A process will be initiated aimed at publishing an RFC specifying the
Memento solution.

•  TimeGate URI syntax convention for Web archives will be discussed
in the context of International Internet Preservation Consortium.

160

Towards Acceptance …

161

Towards Acceptance: Hyping

162

Towards Acceptance: Discussing

http://efoundations.typepad.com/
efoundations/2009/11/memento-and-
negotiating-on-time.html

Is the Memento proposal OK from the
perspective of the Architecture of the World
Wide Web, and REST?

Resulted in some adjustments to the
proposed framework.

163


http://www.mementoweb.org/events/
IA201002/

Two day discussion with Internet Archive,
California Digital Library, LoCKSS, Library
of Congress about Memento.

proposed framework.

Resulted in commitment by all represented
archives to deploy experimental Memento
support.

164


http://groups.google.com/group/memento-
dev

Discussion list regarding Memento

proposed framework.

165

Towards Acceptance: Thinking

http://www.open.ac.uk/blogs/telstar/
2009/11/24/the-when-of-the-web/

Use Memento to go from formal citation
of Web page (URL, datetime) to
appropriate archival version of the Web
page.

166

Towards Acceptance: Developing

•  Apache plug-in rule that adds Link header to TimeGate on HTTP
HEAD/GET requests to Original Resources.
•  http://mementoweb.org/tools/apache

•  Memento plug-in for the MediaWiki platform.
o  http://www.mediawiki.org/wiki/Extension:Memento

o  Working towards community acceptance.

o  Started similar process for Drupal.

167

Towards Acceptance: Developing

•  Created demonstration client:
o  FireFox plug-in: http://www.mementoweb.org/tools/ (many

thanks to Sam Adams, Cambridge University)
o  Currently working on plug-in for Internet Explorer

168

Towards Acceptance: Ongoing

•  JISC funded developer contest for the creation of Memento-
based prototype demonstrators: http://wiki.2010.dev8d.org/w/
Talk_5

•  The Memento Team will engage in a funded, collaborative
agreement with the Library of Congress for:
o  Standardization (I-D => RFC).

o  Outreach.

o  Research.

o  Software development.

169

Some Interesting Consequences of Memento …

170

Memento and Resource Persistence (1)

•  URI-R vanishes, but the server that used to serve it is still
operational:

o  The server should still include a HTTP Link header with
rel="timegate" pointing at a TimeGate for URI-R,
irrespective of whether the client issues a DT-conneg request
or not. This allows seamless access to a Memento of URI-R,
even if the server no longer hosts the original.
o  If the server does not include a HTTP Link header pointing
at a TimeGate for URI-R, the client resorts to interaction with
archives (or with an Aggregator) and arrives at the most
recent Memento of the resource.

171


•  A domain vanishes:

o  The client is looking for a current representation of URI-R that
was hosted by the domain, but fails.
o  The client resorts to interaction with archives (or with an
Aggregator) and arrives at the most recent Memento of the
resource.

172


•  A domain is taken over by a new custodian:

o  The new custodian adheres to other policies regarding which
archive to redirect a DT-conneg request.
o  The client understands from the first-memento/last-
memento HTTP Link header of that archive of choice, that
it does not cover the time range in which the previous
custodian operated the domain.
o  The client resorts to interaction with other archives (or with an
Aggregator) and arrives at an appropriate Memento.

173

Memento and Resource Versioning (1)

Resource state may evolve over time. Requiring a
URI owner to publish a new URI for each change in
resource state would lead to a significant number
of broken references. For robustness, Web
architecture promotes independence between an
identifier and the state of the identified resource.

From: The Architecture of the World Wide Web, http://
www.w3.org/TR/webarch/

174


But there are many use cases that require resource
versioning, including community-based content
creation, scientific communication, open
government, etc.

How does resource versioning work currently? How
can it work in a Memento Web?

Paper at http://arxiv.org/abs/1003.3661
175

Current Resource Versioning (3)

•  Version Identification: HTTP
URIs

•  Versioning Strategy: new
URI for new version

•  Version Relationships: RDF
or HTTP Links

•  Version Datetime: RDF

176


•  Version Identification: HTTP
URIs

•  Versioning Strategy: stable ,
cool URI for "current" version;
new URI for old version

•  Version Relationships:
HTTP Links, HTTP datetime
conneg

•  Version Datetime: HTTP
Content-Datetime response
header

177

Memento and Non-Information Resources (1)

Current
description
of URI-R

HTTP
Non-Information URI-R 303 URI-S
Information
resource resource

178


Current
description
of URI-R

HTTP
Non-Information URI-R 303 URI-S
Information
resource resource

HTTP LINK HEADER Past
timegate Description
of URI-R

DT-conneg
TimeGate URI-G URI-M Memento

179


•  DBpedia subject URI e.g. http://dbpedia.org/resource/France
leads to current description of the subject.
•  Archive of prior descriptions of DBpedia subject URIs
implemented at LANL:
o  Archive exposes a TimeGate per DBpedia subject URI e.g.

http://mementoarchive.lanl.gov/dbpedia/timegate/http://
dbpedia.org/resource/France
o  These TimeGates support DT-conneg.

•  DBpedia provides HTTP Link header with rel="timegate"
pointing at these TimeGates.
•  Result: Clients can access current and prior descriptions using
"follow your nose" HTTP navigation.
•  Proof-of-Concept: "follow your nose" time-series analysis across
DBpedia versions.

180


Time-Series analysis across DBpedia versions
181

Memento and Time-Persistent Web Annotations

182

Memento wants to make navigating the Web’s Past Easy

http://www.mementoweb.org
http://groups.google.com/group/memento-dev
183

Memento: Updated Technical Details (March 2010)

Recomendados

Recomendados

Mais conteúdo relacionado

Mais de Herbert Van de Sompel

Mais de Herbert Van de Sompel (20)

Último

Último (20)

Memento: Updated Technical Details (March 2010)