hn.today

British Library Legal deposit and web archiving

bl.uk3 points0 comments
Screenshot of British Library Legal deposit and web archiving

The British Library and the other Legal Deposit Libraries can archive material published on the UK web under legal deposit law. Each year, the UK Web Archive aims to capture a snapshot of identifiable UK websites - at least four million sites and several billion files. A site may qualify through a UK-related domain, UK contact address, evidence that its content was created in the UK, or a server hosted in the UK. The archive excludes content shared only in restricted private groups and material consisting solely of sound recordings or moving images; it may include audio or video when accompanying text or other content has independent meaning.

Websites are usually collected by automated crawlers, which start from selected URLs and follow links to gather pages, files and metadata. Crawlers limit the depth and volume of requests and leave intervals between requests to avoid overloading sites. They identify themselves to web servers, but cannot access material behind logins without credentials. After disruption in 2023, the UK Web Archive has been restored, but access remains limited: only material archived before December 2022 is currently searchable, collections are static, and users must enter a full URL to explore a site. Most archived content can be viewed only in a Legal Deposit Library reading room, so visitors should check availability before travelling. The collecting programme is governed by the Deposit Libraries Act 2003 and related 2013 regulations; UK websites can be nominated for inclusion.

Read on bl.uk0 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in Web

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.