World Library  
Flag as Inappropriate
Email this Article

Archive-It

Article Id: WHEBN0028741374
Reproduction Date:

Title: Archive-It  
Author: World Heritage Encyclopedia
Language: English
Subject: Internet Archive
Collection:
Publisher: World Heritage Encyclopedia
Publication
Date:
 

Archive-It

Not to be confused with arXiv.
For help citing the Internet Archive in English World Heritage Encyclopedia, see WP:Using the Wayback Machine.

Coordinates: 37°46′56.3″N 122°28′17.65″W / 37.782306°N 122.4715694°W / 37.782306; -122.4715694

Internet Archive
Type 501(c)(3) non-profit
Foundation date 1996 (1996)
Headquarters Richmond District, San Francisco, California, USA
Chairman Brewster Kahle
Services Archive-It, Open Library, Wayback Machine (since 2001), Netlabels, NASA Images, Prelinger Archives
Employees 200
Slogan(s) Universal access to all knowledge
Website
Alexa rank positive decrease 163 (November 2013)[1]
Type of site Digital library
Available in English
Launched 2001

Headquarters

Since 2009, headquarters have been at 300 Funston Avenue in San Francisco, a former Christian Science Church.
From 1996 to 2009, headquarters were in the Presidio of San Francisco, a former U.S. military base.


The Internet Archive is a non-profit digital library with the stated mission of "universal access to all knowledge."[2][3] It provides permanent storage of and free public access to collections of digitized materials, including websites, music, moving images, and nearly three million public-domain books. As of October 2012, its collection topped 10 petabytes.[4][5] In addition to its archiving function, the Archive is an activist organization, advocating for a free and open Internet.

The Internet Archive allows the public to upload and download digital material to its data cluster, but the bulk of its data is collected automatically by its web crawlers, which work to preserve as much of the public web as possible. Its web archive, The Wayback Machine, contains over 150+ billion web captures.[6][7] The Archive also oversees one of the world's largest book digitization projects.

Founded by Brewster Kahle in 1996, the Archive is a 501(c)(3) non-profit operating in the United States. It has an annual budget of $10 million, derived from a variety of sources: revenue from its Web crawling services, various partnerships, grants, donations, and the Kahle-Austin Foundation.[8] Its headquarters are in San Francisco, California, where about 30 of its 200 employees work. Most of its staff work in its book-scanning centers. The Archive has data centers in San Francisco, Redwood City, and Richmond, all in California. Its collection is mirrored for stability and endurance at the Bibliotheca Alexandrina in Egypt.[9]

The Archive is a member of the International Internet Preservation Consortium.[10] This non-profit digital library was officially designated as a library by the State of California in 2007.[11]

History

Kahle founded the Archive in 1996 at the same time that he began the for-profit web crawling company Alexa Internet. In 1996, The Internet Archive had begun to archive and preserve the World Wide Web. The archived content wasn't available until 2001, when it developed the Wayback Machine. In late 1999, the Archive expanded its collections beyond the Web archive, beginning with the Prelinger Archives. Now the Internet Archive includes texts, audio, moving images, and software. It hosts a number of other projects: the NASA Images Archive, the contract crawling service Archive-It, and the wiki-editable library catalog and book information site Open Library. Recently, the Archive has begun working to provide specialized services relating to the information access needs of the print-disabled; publicly accessible books were made available in a protected Digital Accessible Information System (DAISY) format.[12]

According to its website:

Most societies place importance on preserving artifacts of their culture and heritage. Without such artifacts, civilization has no memory and no mechanism to learn from its successes and failures. Our culture now produces more and more artifacts in digital form. The Archive's mission is to help preserve those artifacts and create an Internet library for researchers, historians, and scholars.

In August 2012, the Archive announced[13] that it has added BitTorrent to its file download options for over 1.3 million existing files, and all newly uploaded files.[14][15] This method is the fastest means of downloading media from the Archive, as files are served from two Archive datacenters, in addition to other torrent clients which have downloaded and continue to serve the files.[14][16]

Wayback Machine

Main article: Wayback Machine



The Internet Archive has capitalized on the popular use of the term "WABAC Machine" from a segment of the old Rocky and Bullwinkle cartoon, and uses the name "Wayback Machine" for its service that allows archives of the World Wide Web to be searched and accessed.[17] This service allows users to view archived web pages. The Wayback Machine was created as a joint effort between Alexa Internet and the Internet Archive when a three dimensional index was built to allow for the browsing of archived web content.[18] Millions of websites and their associated data (images, source code, documents, etc.) are saved in a gigantic database. The service can be used to see what previous versions of websites used to look like, to grab original source code from websites that may no longer be directly available, or to visit websites that no longer even exist. The Internet Archive Terms of Use specify that users of the Wayback Machine are not to download data from the collection. Not all websites are available because many website owners choose to exclude their sites. As with all sites based on data from web crawlers, the Internet Archive misses large areas of the web for a variety of other reasons. International biases have also been found in its coverage, although this does not seem to be the result of a deliberate policy.[19]

Examples from the Wayback
Machine's archives:
  • Apple Computer
  • Amazon
  • Microsoft
  • BBC News
  • Google
  • Open Directory
  • World Heritage Encyclopedia

The use of the term "Wayback Machine" in the context of the Internet Archive has become so common that "Wayback Machine" and "Internet Archive" are almost synonymous. This usage occurs in popular culture, e.g., in the television show Law and Order: Criminal Intent ("Legacy", first run Aug. 3, 2008), an extra playing a computer tech uses the "Wayback Machine" to find an archive of a student's Facebook style website. Snapshots usually take at least 6–18 months to be added.

Open Library

Main article: Open Library


The Open Library is another project of the Internet Archive. The site seeks to include a web page database for every book ever published, a sort of open source version of WorldCat. It holds 23 million catalog records of books, in addition to the full texts of about 1,600,000 public domain books, which are fully readable and downloadable.[20][21] Open Library is a free/open source software project, with its source code freely available on the Open Library site.

Archive-It

File:Internet-archive-brewster-kahle-2013-0329.webm

Created in early 2006, Archive-It[22] is a web archiving subscription service that allows institutions and individuals to build and preserve collections of digital content and create digital archives. Archive-It allows the user the option to customize their capture or exclusion of web content they want to preserve for cultural heritage reasons. Through a web application, Archive-It partners can harvest, catalog, manage, browse, search and view their archived collections. In terms of accessibility, the archived websites are full text searchable within seven days of capture.[23] Content collected through Archive-It is captured and stored as a WARC file. A primary and back-up copy is stored at the Internet Archive data centers. A copy of the WARC file can be given to subscribing partner institutions for geo-redundant preservation and storage purposes to their best practice standards.[24] The data captured through Archive-It is periodically indexed into the Internet Archive's general archive.

As of December 2012, Archive-It had over 225 partner institutions in 45 U.S. States and 15 countries that have captured over 4.9 billion URLs for over 1974 public collections. Archive-It partners are universities and college libraries, state archives, federal institutions, museums, law libraries and cultural organizations, including the Electronic Literature Organization, North Carolina State Archives and Library, Stanford University, Columbia University, American University in Cairo, Georgetown Law Library and many others.

NASA Images

NASA Images was created through a Space Act Agreement between the Internet Archive and NASA to bring public access to NASA's image, video, and audio collections in a single, searchable resource. The NASA Images team works closely with all of the NASA centers to keep adding to the ever-growing collection at NASA Images[25] The site launched in July 2008 and now has more than 100,000 items online.

Media collections


In addition to web archives, the Internet Archive maintains extensive collections of digital media that are attested by the uploader to be in the public domain in the United States or licensed under a license that allows redistribution, such as Creative Commons licenses. Media are organized into collections by media type (moving images, audio, text, etc.), and into sub-collections by various criteria. Each of the main collections includes an "Open Source" sub-collection where general contributions by the public are stored.

Moving image collection

The Internet Archive holds a collection of approximately 3,863 feature films.[26] Aside from feature films, the Internet Archive's Moving Image collection includes: newsreels; classic cartoons; pro- and anti-war propaganda; The Video Cellar Collection; Skip Elsheimer's "A.V. Geeks" collection; and ephemeral material from Prelinger Archives, such as advertising, educational and industrial films and amateur and home movie collections.

Subcategories of this collection include:

IA's September 11, 2001 as they unfolded on live television.

Some of the films available on the Internet Archive are:[27]

Links to the online film are in the External links section for each article. See also World Heritage Encyclopedia list of films freely available on the Internet Archive.

Machinima archive

One of the sub collections of the Internet Archive's Video Archive is the Machinima Archive. This small section hosts many Machinima videos (see Machinima: Virtual Filmmaking). Machinima is a digital artform in which computer games, game engines or software engine are used in a sandbox mode like mode to create motion pictures, recreate plays or even publish presentations/keynotes. The archive collects a range of Machinima films from internet publishers such as Rooster Teeth and Machinima.com as well as independent producers. The sub collection is a collaborative effort between the Internet Archive, the How They Got Game research project at Stanford University, the Academy of Machinima Arts and Sciences and Machinima.com.[28]

TV News Search & Borrow


In September 2012, the Internet Archive launched the TV News Search & Borrow[29] service for searching U.S. national news programs. The service is built on closed captioning transcripts and allows user to search and stream 30-second video clips. Upon launch, the service contained "350,000 news programs collected over 3 years from national U.S. networks and stations in San Francisco and Washington D.C."[30] According to Kahle, the service was inspired by the Vanderbilt Television News Archive, a similar library of televised network news programs.[31] In contrast to Vanderbilt, which limits access to streaming video to individuals associated with subscribing colleges and universities, the TV News Search & Borrow allows open access to its streaming video clips.

Audio collection

Main article: Live Music Archive

The Audio Archive includes music, audio books, news broadcasts, old time radio shows and a wide variety of other audio files. There are over 200,000 free digital recordings in the collection. The subcollections include audio books and poetry, podcasts, non-English audio and many others.[32]

The Live Music Archive sub-collection includes over 100,000 concert recordings from independent artists, as well as more established artists and musical ensembles with permissive rules about recording their concerts such as the Grateful Dead, and more recently, The Smashing Pumpkins. Also, Jordan Zevon has allowed Internet Archive to host a definitive collection of his father Warren Zevon concert recordings. The catalog ranges from 1976-2001 and contains 1,137 free songs.[33]

Netlabels

Not to be confused with Netlabel.

The Archive has a collection of freely distributable music that is downloadable and streamable via its Netlabels service. The music in this collection generally have Creative Commons-license catalogs of virtual record labels.[34][35]

Text collection


The Internet Archive Text Archive collection includes digitized books and special collections from various libraries and cultural heritage institutions from around the world. The Internet Archive operates twenty-three scanning centers in five countries, digitizing about 1,000 books a day, financially supported by libraries and foundations.[36] As of November 2008, when there were about 1 million texts, the entire collection was over 0.5 petabytes, which includes raw camera images, cropped and skewed images, PDFs, and raw OCR data.[37]

Between about 2006 and 2008 Microsoft Corporation had a special relationship with Internet Archive texts through its Live Search Books project, scanning over 300,000 books which were contributed to the collection, as well as financial support and scanning equipment. On May 23, 2008, Microsoft announced it would be ending the Live Book Search project and no longer scanning books.[38] Microsoft made its scanned books available without contractual restriction and donated its scanning equipment to its former partners.[38]

Around October 2007 Archive users began uploading public domain books from Google Book Search.[39] As of May 2011 there were over 900,000 Google-digitized books in the Archive's collection, out of a total of 2.8 million books. The books are identical to the copies found on Google, except without the Google watermarks, and are available for unrestricted use and download.[40]

Physical media


Voicing a strong reaction to the idea of books simply being thrown away, and inspired by the Svalbard Global Seed Vault, Kahle now envisions collecting one copy of every book ever published. "We're not going to get there, but that's our goal," he said. Alongside the books, Kahle plans to store the Internet Archive's old servers, which were replaced in 2010.[41]

Software


The Internet Archive is «the largest collection of historical software online in the world», spanning 50 years of computer history in terabytes of computer magazines and journals, books, shareware discs, FTP websites, video games etc.[42] In 2013 the Internet Archive began to provide abandonware video games browser-playable via MESS, for instance the Atari 2600 game E.T. the Extra-Terrestrial.[43]

Controversies and legal disputes

Omni magazine

In a story at his Web site headed "What the heck is going on at Internet Archive?", author Steven Saylor noted, “Sometime in 2012, the entire run of Omni magazine was uploaded (and made downloadable) at Internet Archive.... Since those old issues must contain hundreds of works still under copyright by numerous contributors, how is this legal?"[44] At least one contributor to the magazine, author Steve Perry, has publicly complained that he never gave permission for his work to be uploaded ("they didn't say a word in my direction"),[45] and it has been noted that all issues containing the work of Harlan Ellison have apparently been taken down.[46] Glenn Fleishmann, investigating the question "Who Owns Omni?", writes that "Almost all of the authors, photographers, and artists whose work appeared in the magazine had signed contracts that granted only short-term rights....[No one] could simply reprint or post the content from older issues."[47]

Scientology

In late 2002, the Internet Archive removed various sites critical of Scientology from the Wayback Machine.[48] The error message stated that this was in response to a "request by the site owner."[49] It was later clarified that lawyers from the Church of Scientology had demanded the removal and that the actual site owners did not want their material removed.[50]

Healthcare Advocates, Inc.

In 2003, Harding Earley Follmer & Frailey defended a client from a trademark dispute using the Archive's Wayback Machine. The lawyers were able to show that the plaintiff's claims were invalid based on the content of their web site from several years prior. The plaintiff, Healthcare Advocates, then amended their complaint to include the Internet Archive, accusing the organization of copyright infringement as well as violations of the DMCA and the Computer Fraud and Abuse Act. Healthcare Advocates claimed that, since they had installed a robots.txt file on their website, even if after the initial lawsuit was filed, the Archive should have removed all previous copies of the plaintiff website from the Wayback Machine.[51] The lawsuit was settled out of court.[52]

Robots.txt is used as part of the Robots Exclusion Standard, a voluntary protocol the Internet Archive respects that disallows bots from indexing certain pages delineated by the creator as off-limits. As a result, the Internet Archive has rendered unavailable a number of websites that are now inaccessible through the Wayback Machine. Currently, the Internet Archive applies robots.txt rules retroactively; if a site blocks the Internet Archive, like Healthcare Advocates, any previously archived pages from the domain are also rendered unavailable. In cases of blocked sites, only the robots.txt file is archived.

However, the Internet Archive also states, "Sometimes a web site owner will contact us directly and ask us to stop crawling or archiving a site. We comply with these requests."[53] In addition, the website says: "The Internet Archive is not interested in preserving or offering access to Web sites or other Internet documents of persons who do not want their materials in the collection."[54]

Suzanne Shell


On December 12, 2005, activist Suzanne Shell demanded Internet Archive pay her US$100,000 for archiving her website profane-justice.org between 1999 and 2004.[55] Internet Archive filed a declaratory judgment action in the United States District Court for the Northern District of California on January 20, 2006, seeking a judicial determination that Internet Archive did not violate Shell’s copyright. Shell responded and brought a countersuit against Internet Archive for archiving her site, which she alleges is in violation of her terms of service.[56] On February 13, 2007, a judge for the United States District Court for the District of Colorado dismissed all counterclaims except breach of contract.[55] The Internet Archive did not move to dismiss copyright infringement claims Shell asserted arising out of its copying activities, which will also go forward.[57]

On April 25, 2007, Internet Archive and Suzanne Shell jointly announced the settlement of their lawsuit. The Internet Archive said, “Internet Archive has no interest in including materials in the Wayback Machine of persons who do not wish to have their Web content archived. We recognize that Ms. Shell has a valid and enforceable copyright in her Web site and we regret that the inclusion of her Web site in the Wayback Machine resulted in this litigation. We are happy to have this case behind us.” Shell said, “I respect the historical value of Internet Archive’s goal. I never intended to interfere with that goal nor cause it any harm.”[58]

Grateful Dead


In November 2005, free downloads of Grateful Dead concerts were removed from the site. John Perry Barlow identified Bob Weir, Mickey Hart, and Bill Kreutzmann as the instigators of the change, according to a New York Times article.[59] Phil Lesh commented on the change in a November 30, 2005, posting to his personal website:

It was brought to my attention that all of the Grateful Dead shows were taken down from Archive.org right before Thanksgiving. I was not part of this decision making process and was not notified that the shows were to be pulled. I do feel that the music is the Grateful Dead's legacy and I hope that one way or another all of it is available for those who want it.[60]

A November 30 forum post from Brewster Kahle summarized what appeared to be the compromise reached among the band members. Audience recordings could be downloaded or streamed, but soundboard recordings were to be available for streaming only. Concerts have since been re-added.[61]

National security letter

File:EFF-IA National security letter.pdf
File:EFF-IA National security letter.pdf
A National security letter issued to the Internet Archive demanding information about a user

On May 8, 2008, it was revealed that the Internet Archive successfully challenged an FBI National Security Letter asking for logs on an undisclosed user.[62][63]

Uncensored hosting

On August 17, 2011, Middle East Media Research Institute published "Al-Qaeda, Jihadis Infest the San Francisco, California-Based 'Internet Archive' Library"[64] which detailed how members can post anonymously and enjoy free uncensored hosting.

Opposition to Google Books settlement

The Internet Archive is a member of the Open Book Alliance, which has been among the most outspoken critics of the Google Book Settlement. The Archive advocates an alternative digital library project.[65]

Opposition to SOPA and PIPA bills

The Internet Archive blacked out its website for twelve hours on 18 January 2012 in protest of the Stop Online Piracy Act and the PROTECT IP Act bills, two pieces of pending legislation in the United States Congress that they claim will "negatively affect the ecosystem of web publishing that led to the emergence of the Internet Archive." This occurred in conjunction with the English World Heritage Encyclopedia blackout, as well as numerous other protests across the internet.[66]

Ceramic Archivists Collection

The Great Room of the Internet Archive features a collection of nearly 100 ceramic figures representing employees of the Internet Archive. This collection was commissioned by Brewster Kahle and is ongoing. The sculptor is Nuala Creed.

List of additional collections

Collection Number of items
(April 11, 2013)
Number of Sub-Collections
(April 14, 2013)
Arpanet 195 0
Dance Manuals 208 0
Aljam3a-Library المكتبة الجامعة 12,960 0
Ant Texts 2,555 0
Aozora Bunko 青空文庫 4,000 0
Bishop's College School 167 0
Boletin Oficial de la Republica Argentina 25,376 0
Building Technology Heritage Library 3,867 0
BYU Utah Valley Obits 8,973 0
California Garden 673 0
Charity Hospital Reports 115 0
Classic Comics 729 0
Collected texts of Simon Schwartzman 884 0
Cookbooks and Home Economics 3,018 0
Des Moines and Polk County, Iowa, City Directories:1866-1922 43 0
Digital Fanzine Preservation Society 72 0
dwsolo 66 0
Flat World Knowledge Creative Commons Archive 119 0
Folkscanomy: A Library of Books 689 6
Genealogy 77,986 16
Grammar Analysis 355 0
International Music Score Library Project 26,680 0
JSTOR Early Journal Content 452,019 342
Make Projects 195 0
Mathematics Library[67] 272 4
Microfilm[68] 160,271 2
No Borders Bookstore 66 0
No Borders Bookstore - 0
Open Reading Room 1,912 0
Opensource Textbooks 2,207 14
Panama-California Exposition Digital Archive 105 0
Philosophical Transactions of the Royal Society of London 18,553 0
Prelinger Library 3,809 1
Pythagoras Publishing 287 0
Quigley / Reynolds Family History 62 0
RECAP US Federal Court Documents[69] 892,321 0
RECAP US Federal Court Documents (Development) 11 0
Russian History 37 0
Shimer Alumni (Unauthorised) Archive 255 0
Siyavula Education Collection 166 0
Social Sciences Society, HKUSU - SSS Library 143 0
Solidarity! Revolutionary Center and Radical Library 836 0
Tamil Nadu Text Books 260 0
The Beat Within Magazine 131 0
The BITSAVERS.ORG Documents Library 8,741 25
The Collection Of The International Psychoanalytic University Berlin 821 0
The Crittenden Automotive Library 842 0
The Harvard Classics / Dr. Eliot's Five Foot Shelf 159 0
The Science Fiction and Fantasy Fiction Collection 315 0
The Swedish National Museums of World Culture 35 4
The Tibetan Buddhist Resource Center Library 2 0
The University of Wisconsin-Stout Archives 99 0
The Zamorano 80 146 0
US Government Documents 32,365 6
USGS Water Resources of New York Reports 107 0
Webcomic Universe 5,219 0
WWII Archive 1,585 0
École des Ponts ParisTech 31 0

See also

Similar projects

Other

References

Further reading

  • Kahle, Brewster, "Archiving the Internet", November 4, 1996
  • Ringmar, Erik, "Liberate and Disseminate," Times Higher Education Supplement, April 10, 2008.

External links

  • in California, United States
  • Internet Archive Mirror at the Bibliotheca Alexandrina, Egypt
  • Netlabels release collection
  • radiom.org
  • Web Archiving at archive.org, details of Internet Archive operations
  • Pictures and descriptions of the Wayback Machine hardware in 2003 (prior to the Petabox), with cost information
  • Current Petabox storage hardware
  • Earliest known website of Internet Archive (www.archive.org) from 1997
  • Early websites from 1996
This article was sourced from Creative Commons Attribution-ShareAlike License; additional terms may apply. World Heritage Encyclopedia content is assembled from numerous content providers, Open Access Publishing, and in compliance with The Fair Access to Science and Technology Research Act (FASTR), Wikimedia Foundation, Inc., Public Library of Science, The Encyclopedia of Life, Open Book Publishers (OBP), PubMed, U.S. National Library of Medicine, National Center for Biotechnology Information, U.S. National Library of Medicine, National Institutes of Health (NIH), U.S. Department of Health & Human Services, and USA.gov, which sources content from all federal, state, local, tribal, and territorial government publication portals (.gov, .mil, .edu). Funding for USA.gov and content contributors is made possible from the U.S. Congress, E-Government Act of 2002.
 
Crowd sourced content that is contributed to World Heritage Encyclopedia is peer reviewed and edited by our editorial staff to ensure quality scholarly research articles.
 
By using this site, you agree to the Terms of Use and Privacy Policy. World Heritage Encyclopedia™ is a registered trademark of the World Public Library Association, a non-profit organization.
 


Copyright © World Library Foundation. All rights reserved. eBooks from Project Gutenberg are sponsored by the World Library Foundation,
a 501c(4) Member's Support Non-Profit Organization, and is NOT affiliated with any governmental agency or department.