The networking of free-access Legal Information Institutes (LIIs) has resulted in a network of 15 LIIs providing free access to over 700 databases from 95 countries. Through the largest portal to the network, the World Legal Information Institute (WorldLII), all 700+ databases can be searched simultaneously. The number of databases in the network is growing by about 100 per year at present. While the databases from many of the countries are quite small, they are very substantial from others. From Canada, Australia, Hong Kong, India, Papua New-Guinea, the Philippines, South Africa, Ireland, the UK, New Zealand and 20 Pacific Island countries, they are comparable to what is available from some commercial providers. Before 2001 there were some LIIs, but no network. This article attempts a brief account of what has happened to provide an alternative to commercial publishers’ networks.

International networking of legal information – commercial publishers

When did legal information systems first contain significant content from multiple countries? By the early1980s Lexis (now LexisNexis) had added UK and French content to its dial-up service. EURONET Diane also contained legal databases from a number of countries (Bing 1984).

By the mid-80s other content from Europe, New Zealand and Australia was available on LexisNexis. Today, this has expanded to cover content from 19 countries, under the LexisNexis ‘Global Legal’ section (previously less polite as ‘Non-US’). Thirteen are Commonwealth countries, and ‘Commonwealth’ is the only multi-country search provided. Provision of caselaw and legislation varies. Caselaw and legislation cannot be searched together so it is impossible to simultaneously search the whole of LexisNexis.

Westlaw International has gradually expanded since 2000 to cover caselaw and legislation from six jurisdictions: the United States, United Kingdom, the European Union, Australia, Hong Kong, and Canada (Westlaw 2007). The maximum scope of a simultaneous search is 10 databases, and you need to be able to select 13 to search everything they have. Cases and legislation can be searched simultaneously, with other content. The third potential commercial network has not yet happened. The Wolters Kluwer/CCH companies have content scattered on sites around the world (US, UK, CCH Asia, Australia, New Zealand, Canada etc) but no network or mega-site.

The LexisNexis and Westlaw International models were based on the model of one centralized location of data from multiple countries, and remain that way. In the mid-1980s there were attempts to establish another model: a dial-up network of collaborating commercial providers of legal information from multiple countries. For example, Australia’s then-CLIRS system (later bought by LexisNexis) tried to set up a network with the UK’s Eurolex and Canada’s QL, but this was scuttled when LexisNexis bought Eurolex, despite litigation challenging this (see Greenleaf, Mowbray and Lewis, 1988, p37). Such networks were in any event gateways at best, not allowing for simultaneous searching. During the CD-ROM era of legal research, from about 1988-95, it seems the emphasis was on circulating plastic, not building networks.

Enter the LIIs – a third network emerges

The Legal Information Institute started at Cornell in 1992 with a number of databases primarily of US federal law, and was the first significant source of free access law on the Internet. The Australasian Legal Information Institute (AustLII) was started by UNSW and UTS Law Schools in1995. It borrowed the ‘LII’ suffix from Cornell, as others have done since, but was the first LII to attempt to build a comprehensive free access national legal information system rivalling that of commercial publishers.

From 1999 AustLII started to use its search engine (Sino) and other software to assist organisations in other countries to establish LIIs with similar functionality and usually with academic roots: in 2000, the British & Irish Legal Information Institute (BAILII), based at the Institute of Advanced Legal Studies, London and University of Cork; in 2001, the Pacific Islands Legal Information Institute (PacLII) at the University of the South Pacific Faculty of Law for fourteen island countries of the Pacific (now 20); in 2002 the Hong Kong Legal Information Institute (HKLII) at Hong Kong University; the Southern African Legal Information Institute (SAFLII) with Wits University Law school in 2003, now operated by the South African Constitutional Court Trust for sixteen countries in southern and eastern Africa; and the New Zealand Legal Information Institute (NZLII) established by Otago University Law School in 2004. For all these LIIs, AustLII established the initial system and ran it for a while, with the aim of assisting full local take-over as soon as possible. Cylaw in Cyprus developed independently from 2004, but using AustLII’s search engine.

Meanwhile, the long-established LexUM team at the University of Montreal established the Canadian Legal Information Institute (CanLII) in 2000, initially by using the AustLII software but then developing their own. They also developed Juri Burkina (2003), Droit Francophone (2003), and are assisting developments in other countries.

BAILII and PacLII were multi-country regional systems from inception, but did not involve material from other LIIs. A linguistic rather than regional focus was taken by Droit Francophone (2003), with initial content concentrating on West and Central Africa, but some databases from across the francophonie. It is a single LII but now includes content derived from Juri Burkina. WorldLII at www.worldlii.org (2002) was the first multi-LII site, initially incorporating data from AustLII, BAILII, PacLII, HKLII and CanLII. Initially, the concordances for all the databases were located on the WorldLII server in Sydney, except that searches were sent to CanLII, and the returned results merged with the locally-produced results from the other LIIs. The latency of the CanLII results often meant that users had to presented results stating ‘CanLII results not available’.

Two multi-LII sub-portals followed. In 2005 AustLII developed the Commonwealth Legal Information Institute (CommonLII, 2005) for Commonwealth and Common Law countries (‘droit Anglophone’ is its nickname), which added databases from 20 new countries, but also relied upon the content of existing LIIs: AustLII, BAILII, CanLII, PacLII, HKLII, SAFLII and CyLaw. The Asian Legal Information Institute (AsianLII, 2006), drew on CommonLII, PacLII (for PNG) and HKLII, but also databases from 12 additional Asian countries. All of this fed back into WorldLII.

An institutional network – Free Access to Law

The Free Access to Law Movement is the institutional equivalent of the LII network. It is a loose affiliation of all the LIIs mentioned above and a number of other organisations with similar goals. It meets annually if possible during the ‘Law via Internet’ Conference (see LVI Conferences), and by email between Conferences. It was formed in 2002 and adopted the Declaration on Free Access to Law, which has had some amendments since then (Declaration 2007). Its website is being constructed at www.freelii.org. Membership is by invitation, with members nominating new candidates, and consensus required. The membership criteria are not fixed but involve adherence to, and support of, the Declaration and activities similar to (but not necessarily identical with) a LII. Recent members are the Institute of Theory of Techniques of Legal Information, Italy (ITTIG), and LawPhil, a University-based free access provider from the Philippines.

There are a lot more countries with databases in WorldLII than there are members of the Free Access to Law Movement. PacLII, Droit Francophone, CommonLII and AsianLII have incorporated into the LIIs the legal content of many organisations beyond their ‘home’ jurisdictions. How this is done varies a great deal. Sometimes the local involvement required is quite passive: a court or legislation website indicates that other are allowed to republish its content, or the copyright law of a jurisdiction states there is no copyright in legal documents (eg in Macau or India). In other cases obtaining republication rights requires requests and negotiation, but they are rarely refused if the data is already being published for free access. Otherwise, negotiating the supply of a steady stream of otherwise unobtainable data is sometimes very difficult particularly if a commercial provider is already obtaining it (eg Malaysian legislation). In other cases the institution concerned is pleased to obtain an opportunity to publish on the Internet (eg some Ugandan courts). PacLII and SAFLII have staff members who travel from country to country to obtain data not otherwise available. A discussion of the many complexities of data acquisition would be very lengthy. It is sufficient to say that the LII network has only scraped the surface of the valuable legal information which is potentially available.

The main constraining factor is of course funds. Every LII looks after the funding of its own system. The models on which LIIs are funded vary a great deal. AustLII has a ‘multi-stakeholder’ model, with more than 50 stakeholders in that case (see AustLII funding 2007). BAILII is similar. Most LIIs have had a considerable deal of academic funding and institutional support. CanLII is funded primarily by the Canadian legal profession. Aid agencies have made significant contributions to PacLII, SAFLII and AsianLII. Strategic alliances with some legal publishers have helped AustLII along. A small LII like CyLaw is a personal project. Yet others like NZLII live on ‘the smell of an oily rag’ and a little help from their friends, while they search for longer-term funds. There is no Daddy Warbucks (or even a George Soros) who would fund global free access to law long-term, but that doesn’t mean it can’t be done. It has been done with ever-widening scope for over a decade. There is not one formula, but as with many other aspects of open content, there are many non-business models by which numerous stakeholders can be engaged.

Network of shared databases, multiple interfaces

The LIIs have agreed in principle to allow their content to be shared between members of the Free Access to Law Movement who wish to build different interfaces to its content (‘portals’ in one usage). So far, WorldLII is the only interface to all the LII content. Various LIIs draw on databases contained on other LIIs, as well as on their own content, for presentation in their search results: CommonLII (AustLII, CanLII, PacLII, HKLII, NZLII, Cylaw), AsianLII (CommonLII and PacLII). So, for example, the same databases from India appear in CommonLII, AsianLII and WorldLII, badged as CommonLII. The same databases from Papua New Guinea appear on PacLII, CommonLII, AsianLII and WorldLII, badged as PacLII. Other possibilities are for interfaces that draw on the databases on any LII in a particular language. A Portuguese portal could cover existing databases from Macau and Timor Leste, and some to come from Africa. Droit Francophone could in theory include some French language databases on other LIIs. There are obviously many possibilities.

All of the LII’s operated by AustLII (WorldLII, CommonLII, AsianLII) share the same file structure on AustLII’s servers. For example, all Indian databases are located at …/in/, so www.asianlii.org/in/, www.commonlii.org/in/ and www.worldlii.org/in/ all address the same files. The structure is therefore simple enough to accommodate databases from any country in the world, and any number of different interfaces, without changes.

Google?

Does networking matter in a post-Google world? Cannot search engines just spider everything? Well, no. Most LIIs use the Robot Exclusion Protocol (Robots Exclusion 2007) to exclude spiders from at least their case-law, on privacy grounds (you should not find an old friend’s divorce or criminal case when arranging a school reunion). Networking LIIs can also add many forms of organisation of the data shared between LIIs that general search engines don’t yet provide (Greenleaf 2005). The future may be different, but then it will be a question of who pays for the LII value-adding from which search engines profit.

Technical robustness of the network

The LII network now relies upon a replication/ synchronisation model. A complete copy of all LII data is held in Sydney and is mirrored locally both at UTS and UNSW. WorldLII, CommonLII and AsianLII are built on that structure, with distributed searches no longer necessary to produce search results. Other interfaces to the network built by other LIIs could similarly utilise the local replication of LIIs. LexUM/CanLII and AustLII have both installed additional servers to facilitate their respective replication of the other’s data, so LexUM now controls a server in Sydney and AustLII controls one in Montreal. Data and system setup is synchronised on both systems using rsync on a regular basis. The Montreal system also serves as a North American home for several other systems including PacLII, SAFLII, NZLII, CommonLII, AsianLII and WorldLII.

Other LIIs contribute less extensive replication facilities. The PacLII system is maintained in Vanuatu, but is mirrored in Sydney. Traffic originating from the Pacific (or more accurately, the USP network) is directed to the Vanuatu server, with the rest of the world defaulting to Sydney. Similarly, the SAFLII system is maintained in South Africa but is again mirrored in Sydney. At the moment, all traffic is directed to Sydney, but as larger servers are installed in Johannesburg, load balancing/ failover similar to PacLII will be deployed. BAILII and HKLII are also exploring replication options.

A geographically distributed network of independent LIIs poses significant technical challenges if it is to be robust and to ‘scale up’ successfully. Desirable attributes include replication of data sets across a global network of machines, with synchronisation approaching real-time; flexible load balancing and allocation of requests across the network; and failover access to local machines with replicated copies where the host LII is unavailable. As part of a project to enhance the robustness of the LII network, AustLII has developed a replacement DNS server known as the Load Balancing Daemon (‘LBD’). LBD is currently being deployed on an experimental basis, and is being gradually introduced across the free access to law network (Mowbray et al 2007).

Futures for the free access to law network

All LIIs have challenges in maintaining their own operations, the most common one being that free access cannot be provided at no cost. The main issue for the Free Access to Law Movement as a whole, and for LII networking, is to find the best ways to expand both to make them more inclusive of organisations with similar goals, and a more global range of quality legal information. So far, both are largely Anglophone plus Francophone.

The technical sophistication of the network, and corresponding collaboration between LIIs, has increased a lot recently, and augurs well for long-term robustness. A related challenge is to increase the interconnections between data on different LIIs, beyond common search results to direct links between data. CanLII’s Reflex (CanLII 2007) and work underway at other LIIs recognises the importance of citations to adding value both locally and across the LII network.

An on-the-horizon problem is the relationship between the LIIs and those who wish to build law-oriented services which attract large advertising revenues partly by spidering LII content, but do not bear any of the ongoing costs of building the quality legal information that the LIIs provide. The attitude of LIIs to spidering is an ongoing question, not only on privacy grounds.

The future for the LII network, its interfaces, and the Free Access to Law Movement is to increasingly provide a global alternative to the expanding global reach of the current legal publishing duopoly. In doing so, it can help to provide better access to law in many countries, developed and developing, and can encourage them to join in a global project that supports economic progress, the rule of law and democracy.

Graham Greenleaf, Andrew Mowbray and Philip Chung are Co-Directors of AustLII

References

All the LIIs can be reached from WorldLII at www.worldlii.org

AustLII funding 2007 – ‘Funding Sources’ page www.austlii.edu.au/austlii/sponsors

Declaration 2007 Declaration on Free Access to Law’ 2002-07 www.worldlii.org/worldlii/declaration

LexisNexis (2007) ‘Company History‘ www.lexisnexis.com/presscenter/mediakit/history.asp

LVI Conferences (1997-2005) – ‘Law via the Internet Conferences’ www.austlii.edu.au/austlii/conference

Greenleaf G. ‘Global Legal Research: WorldLII and the Future’ Internet Newsletter for Lawyers, Jan/Feb 2005 www.venables.co.uk/n0501global.htm

Mowbray, A., Greenleaf, G., Chung, P. and Austin, D. (2007) ‘Improving stability and performance of an international network of free access legal information systems’ ARC eRsearch Seminar, Canberra, February 13-14, 2007, available at law.bepress.com/unswwps

Robots Exclusion (2007) – The Web Robots Pages www.robotstxt.org/wc/exclusion.html

Westlaw (2007) Westlaw International ‘Welcome’ page www.westlawinternational.com