I’m presenting about our Endeca implementation at Access this week and I’m in the throes of presentation prep at the moment. I think I’ve got most of the good stuff covered but I thought I’d put the question out there so that I can make sure to address the burning questions and other need-to-know details. I plan on fleshing out the content in a well-annotated presentation (which I’ll link to here for anyone who is interested & not at Access), so if there’s something you’d really like to know about our implementation, or about the process, let me know? Thanks!

Nice, isn’t it? It’s an interface that we’re pretty proud of (I’m on the UI group) and there’s quite a bit of functionality that makes it far sweeter than what we were used to. Some tidbits:

• Our catalogue used to be called “MORRIS” but we dropped the name in favour of, well, “Library Catalogue” (complete with Canadian spelling, of course). We did this based on the fact that we’re pretty much over cutesy names for search tools, but also based on feedback from users.
• It’s browsable like never before, but then that’s what we’ve come to expect from Endeca-powered search, right? The guided navigation on an average search results page has increased functionality enormously, it truly does make searching (and finding!) a pleasure. The interface you see when you first visit the catalogue page also has multiple browsing options, everything from subject, to language, to author, to location (and more). Personally (and this is just my opinion), I’m still torn over the decision to present all those options right up front (before a search has been entered), mostly because I’m not sure that giving users more options is necessarily a good thing. On the other hand, guided navigation is one of the selling points of an Endeca-powered search in the first place, so it seems a shame not to present those options wherever possible. For now, we’ve decided on the latter, mostly based on use case scenarios, but, like I said, I’m still torn – is it useless clutter or useful functionality? I’m looking forward to usability testing to inform this one (we’ve done a bit, we plan to do more).
• One word: typeahead. When the user starts typing a search term, suggestions appear below the search box in drop-down menu format. The suggestions are culled from title, author and subject MARC fields. It’s super useful, but we still need to tweak a couple of things (there’s a tiny bit of lag; we have to re-look at prioritizing the queried fields so that the most suitable suggestions rise to the top; and a couple of minor design issues). The other concern is that during the minimal usability testing we did do, we observed one user waiting for the typeahead suggestions to come up before hitting the “GO” button (or the “Return” key), which made us think that maybe he didn’t realize that he could go ahead and execute the search even if nothing (or nothing appropriate) appeared in the typeahead menu. This might be something that can be cleared up with a couple of design tweaks (adding the word “Suggestions” to the top of the typeahead suggestion list, for example), and will also probably be informed by more usability testing.
• Another word: AJAX. Clicking on any title in the results list will bring up an AJAX layer that contains all the bibliographic data for that item. The cool-factor here is high, but how usable is it? We’re still not sure (or maybe it’s just me who isn’t sure). Currently, we’ve got the layer coded such that it will disappear after a few seconds when you mouse away from it, which can be either extremely useful or annoyingly frustrating. On the other hand, I’m not sure that requiring a click to close the layer is the way to go either (there is a click-to-close button at the top-right corner of the layer for those who don’t want to wait). Then there’s the design element – does it clutter up the screen too much? Would a lighbox effect be useful at all or would that be too over-the-top design-wise? Still work to be done here obviously, and we’ll likely run it all through (you guessed it) usability before making any final decisions.
• Search within results: how did we ever live without this feature? I find myself asking this question every time I help a user with a catalogue search. We’re still experimenting with the placement of the “search within results” function (you have to perform a search to see where we’ve placed it), but in terms of functionality, I pretty much can’t live without it.

Many of these tweaks, as well as more usability testing obviously, will happen over the next few weeks/months, so it’s still very much a work in progress. If you get a chance to give it a whirl, do pop back here and leave a comment, we’d love to know what you think!

Mellon grants to help with these developments – are the Evergreen folks aware of this?
- one thing that Mellon is doing is gave some seed funding to the U of Rochester to develop their ILS & focus on user interface aspects
- Rochester: a pilot to investigate the feasibility of an extensible catalogue – dumps from the catalogue, harvesting from OAI sources, and bringing all that into an index/database and providing a front-end that includes social computing components; thinking of using Lucene for the indexing.

What does the panel think are the next 2-3 steps that the library development community can take to see some of these ideas come to fruition?
- Art Rhyno (Windsor): look at some of the measures used in the open source arena to identify good software projects (Apache foundation); we need a forum to talk to each other more (listserv? involve associations? IFLA?); how do we bring in everyone from the US, Canada, academic, public sectors, etc.
- Mike Rylander (PINES): more involvement from the library community out to the wider world to drive technology standards; library standards are good but limited – Z39.50 is good but not a lot of people want to implement it because it can’t form the basis of innovation, also SRU/SRW are good but the specs are very hard to read; pushing ideas out into the larger development world is good because it will inform rigourous standards; libraries shouldn’t do what they don’t know how to do as far as development goes, so we need to try hard to pull people (and standards) into our development community.
- David Singleton (PINES): if you build it, they will come. If we build this open source ILS, the community will come. The community has to come because that’s what’s needed to make it sustainable (they always said that they didnt want Evergreen to be born, to grow, and to die in Georgie – they always wanted it to spread!).
- Brad LaJeunesse (PINES): technologies in libraries have grown and become more integrated; tech staff need to be involved in every part of the library operation; technology is part of everything we do so we can’t put it in a corner and leave it to a vendor.
- Peter Murray (OhioLINK): the coalescing of expertise across our institutional boundaries; one institution alone can’t see the development of a system like this, we’ve lost individually and collectively the ability to control our own destiny; but there’s a great deal of expertise across our development community (converge in code4lib, hackfest, etc.); the 80-20 rule applies – 80% of all we do is the same (across institutions) so we need to pull together and take a holistic view of what we want to do & where we want to go.
- Alan Darnell (Scholars Portal): moving forward does require an understanding of the past, so as we move forward we need to remember why we did the things we did in the past (rather than thinking we need to start from scratch and begin anew); know our users, what functionality they want, how they use our stuff; know our content, understand what we have, what its value and purpose is and how it can be used; a business case across institutions to look at how much we’re paying for our ILSs and what we could do if we got together

Do you have anything additional to say to directors and senior managers in the audience? What else can we do to help this process along?
- need “blessings” & support from senior managers to forge ahead!
- these sorts of developments need to rank high on the list of priorities
- dialogue! make it OK to talk about open source and what it is (& what it is not)
- creating a culture of innovation in the library – innovation that allows the library to be flexible, respond to user needs, to respond to changes in technology
- take risks!
- when budgets are tight & times are tough, groups can work together to come up with creative solutions; there’s an opportunity at the director level for directors to get together, save money, take a bold step, and take risks! (Scholars Portal has done this for college & university libraries in Ontario).

Question from the panel: What can developers do to assist directors to sell these projects at higher levels? To make them sustainable, etc.?
- briefings about the project in layman’s terms so that directors can take that and use it on their supervisors!
- look at the emotional and rational components of projects (skills sets, soft skills, etc.) because often people’s reluctance to accept technology is fear

What kinds of things would you say to vendors about their current business models? To stay relevant? To remain as partners?
- add value or you’re irrelevant
- add value without breaking standards
- get on board with strict standards compliance (not compliance to a point)
- a new service model: vendors providing commercial support for open source software (like Red Hat does with Linux; LibLime does with Koha).

Scholars Portal
- repatriation of e-journal literature from publishers
- collection of 7500 journals and 10 million full-text articles
- local load of 130 abstracts and index databases representing over 150 million records
- interest in extending this model to ebooks

Why bother?
- archiving
- ease of access: single interface to find content
- capturing the conversation that occurs in scholarly research

The catalogue
- describes an important body of scholarly research and source material
- it’s absence is a huge gap in our effort to represent the scholarly conversation captured only partially in the electronic article literature
- mix of primary and secondary content
- historical coverage
- but the catalogue and e-journals (the tools we use to make e-journal content available) aren’t well intergrated

Back in the day (early 90s), OPACs were hot and journals were staid and boring. But then something happened – journals “left” (prodigal child!); Scholars Portal is focused on bringing back the journal content. Effort to make it modern, relevant, innovative, and user focus. Answer this question honestly: do you consider your OPAC to be a PC or a Mac?!

Scopus Lucene project
- Elsevier was interested in exploring Lucene to index its content (currently they use Fast) – Lucene is open source
- combine Scopus A&I content with full-text articles from Scholars Portal and XMLMARC records from the U of Toronto Catalogue
- Index them all under Lucene and what do you get?

Challenges, Benefits
- authority control vs. relevance ranking
- whole item vs. components (the OPAC is about not the chapter but the book itself)
- surrogate metadata vs. digital objects (electronic resources are true digital objects)
- open content vs. commercial content (electronic publishing, in the current context, has commercial value and needs to be protected with rights management)

Authority control
- important not only for searching but also, maybe more important, for clustering
- in electronic journals there is no consistency in recording author names – varies from journal to journal
- different vocabularies used by different publishers (sometimes only author supplied tags) – so subject access has never been great in ejournals

So how can we bridge the two?

- Scholars Universe (from CSA) is trying to bridge the gap.
- what if we could continue this by taking our authority records in our catalogues and applying it to ejournal content?

Leveraging the strength of the catalogue
- can we match articles and ebooks to print surrogates and then map vocabularies?
- can we see atomatic classification algorithms with authority terms?

Supplementing surrogate records
- many libraries use TOC, cover image, reviews (e.g. Syndetics) to supplement catalogue records – like eye candyafter you’ve gone thru the search process
- is there any way we can make this content supplementary access points? Like searching the reviews?

Elephant and mouse
- mixing surrpgate records with full-text digital objects creates complexities with relevance ranking
- word occurrence weighted against document length
- using traditional relevance ranking algorithms will favour less complete records

Commercial content
- OPACs are open to all
- if we integrate the content how do we make sure certain material is not available to unauthenticated users?
- move to a finer grained rights management when entitlements are complex (like Shibboleth)
- in the era of Google Scholar, can libraries begin pushing the envelope on public access to metadata from commercial sources?

Finding a common playing field
- do we load econtent into the OPAC or do we load OPAC records into econtent services?
– neither fits very well
– OPAC serves as both a resource discovery tool and an inventory tool
– both functions are necessary but not necessarily best combined into a common application

Liberating data from the OPAC
- XML encoding of MARC records and adoption of Unicode makes it easier to use these records in other contexts
- but do we need an XML schema that represents the object and not the cataogue record?
- provides ability to search and re-factor the content for different views to satisfy different information needs

How do we do this?
- search engine technology (e.g. Lucene)
- but also need structured content repository based on XML

XML databases
- emerging class of tech that allows for storage and querying of XML documents in native format
- Xquery allows for search
- Xquery allows for extraction of elements, refactoring these to create new documents, new views

Stupid xquery tricks!
- great demos and visuals for the rest of the session; I’m not sure if Alan is putting his slides online, but if he does, will let you know where they’re at.

(Brad LaJeunesse, Linux Systems Administrator, Public Information Network for Electronic Services (PINES) Consortium; Mike Rylander, Development Consultant, PINES Consortium; David Singleton – Deputy State Librarian Georgia; Julie Walker, Program Director, PINES Consortium)

Overview of PINES
- 44 public library systems
- 252 member libraries
- 123 counties
- over 8 million items in a single bibliographic database

What makes PINES different?
- single card for all PINES libraries
- users may request delivered from any PINES library to local library at no charge
- in 2006 452,000 loans
- PINES libraries agree to a common set of policies and procedures with the goal that users have a consistent experience at any PINES library
- individual libraries continue to do their own collection development

Governance
- executive committee: nine elected reps (library directors) from member libraries
- function-specific subcommittees make policy recommendations
- executive committee meets quartely and as needed

PINES benefits
- for users: increased access to local library collections
- for libraries: the state of Georgia assumes the costs of the automation system
- access not ownership is the key
- economy of scale for the state: annual cost of PINES $1.6 mill; if the libraries had to do it on their own, it would be $15 m

What do users like best?
- do user surveys regularly
- users like the ability to use any library they drive by
- access to a much bigger collection

The Evergreen project
- 5-yr software contract for PINES ended in June 2005
- 2003-4: PINES staff conducted a survey of the library automation marketplace
- at issue: unique needs of a statewide consortium sharing a centralized database and utilizing a statewide library card
- what do PINES libraries need? Enterprise-class relational database, flexible sys admin, granular permissions structure, complex holds matrix, ability to treat member libraries as individual entities, reports designed to correspond to annual reporting requirements
- Evergreen ILS was developed using open source software
- released under GPL
- alpha release debuted July 2005
- beta release in early 2006
- cost comparison:
– server hardware: evergreen=350,000 vs vendor=1.5 m
– hardware support: evergreen=included for 3 years vs vendor=200,000/yr
– software licensing: evergreen=0 vs vendor=200,000/yr
– local tech staff: evergreen=4 vs vendor=2
- fringe benefits: self-sustaining and control their own destiny; they get to decide on development priorities; no more difficulty trying to convince a vendor to develop important features for them; users of the software have direct access to the developers.
- they did focus groups around the state before they built: 800 people attended (librarians? library staff? users? not sure). the notes from these focus groups formed the blueprint for Evergreen.

Evergreen Online Catalogue Features
- streamlined searching from a single search box
- google-like spell-checking and search suggestions
- ability to select specific material formats from the online catalogue’s front page
- added content, including book cover images, reviews, excerpts
- randomized holds that include geographic location as a factor
- my account: change personal login name, change own password, place/cancel/view holds, RSS (for searches too!), “book bag” feature

Demo: http://gapines.org/opac/en-US/skin/default/xml/index.xml
- working on incorporating user reviews
- browse options on the left (ability to expand and collapse these options – good!)
- my account functionality: set defaults like number of results displayed, default font size, default home library, default search. etc.
- “book bag” feature: can save items for later review, can share the book bag with others (in a study group, etc.).
- “shelf browser” to view what’s on the shelf next to the record on the screen – hugely powerful for users! Browse the shelves from home before you go to the library!
- the idea is that Evergreen is a portal for all library user information needs. So far, they have taken the first step in getting there.

Core technologies
- Database: PostgresQL
- Logic/clue languages: C and Perl, Javascript
- Webserver: Apache mod_perl, C modules
- Client side software: XUL
- Server OS: Linux
- Messaging core: Jabber

[snip] I skipped the sys admin bits, sorry! It was thoroughly fascinating but there was no way I could capture it all.

Open Application Program
- a traditional ILS is a catalogue, an opac, a circ system, cataloguing tools (a monolith!)
- they wanted more – they wanted a platform
- they needed a framework that: is easy to cluster, takes care of all the dirty work, have very low overhead, makes writing applications simple, is built on open, standard protocols
- given this, they decided to create components as they were needed to provide solutions on demand – they decided that they didn’t need to anticipate every problem!

They built OpenSRF. Features and benefits
- a framework that can trivially scale from a single server to hundreds on a tiered, redundant fashion (i.e.: there’s never any downtime!)
- manages everything but the application logic, abstracting away everything to a consistent set of method calls
- can easily handle hundreds of transactions per second, per server without any administrator turning off the underlying components
- turns writing applications, including entire ILS modules into a matter of translating business logic into a set of simple Perl or C routines
- leverages existing open standards and open source software to avoid both duplication of effort and component lock-in.

Benefits of all this
- decreased time to go from service prototypes to production implementations
- allows developers to focus on core ILS issues
- increased capacity as needed using any source – no hardware vendor lock-in!
- no single point of failure for any critical system or service
- rolling upgrades – no need to take the system offline to upgrade backend services!

Where do they go from here?
- migration of the six library systems waiting to become part of pines
- develop acquisitions and serials modules
- work with others on a protocol to share info across automation systems (open NCIP)
- want to develop a children’s portal
- online bill paying for users
- enhance social aspects of the catalogue: user ratings, reviews, comments – users have expressed an interest in this but it’s new ground for libraries; how to also protect user confidentiality while doing this?
- complete the spanish translation for the online catalogue
- to use on mobile devices
- possible partnerships with other institutions – anything is possible with enough time and enough money and the right partners!
- have developed a product that can be used in lots of settings and in lots of institutions

Check them out!
- PINES online catalogue: gapines.org
- Evergreen development log: open-ils.org

Overview
- How can we apply what works in other industries to our industry?
- What is an SOA? And why should I care?
- What might a library SOA look like?
- Who else is talking about this?

What is an SOA?
- It’s not about a particular programming language, network protocol
- It’s an architecture for designing systems (programming-language agnostic)
- SOA are not about large applications, they are about discrete business processes (ignores the how and focuses on the what)
- It’s not about starting over – throwing away everything you have now and starting afresh (doomed to failure!)
- It’s about reuse of things that we already have now (orchestrating business processes; turn our software inside-out)
- It’s not ONLY about web services – it’s about web services as a means to an end (think of SOA as a blueprint for building something and the web services as the plumbing)
- It’s not about proprietary systems, it’s about standards and open protocols

Why should we care about SOA?
- The ILS market is imploding (marketplace consolidation, new entrants into the market, market leaders gobbling each other up and chasing the highest-end customers; some of the least demanding customers are dropping off and going to smaller automation firms; monolithic do-everything applications that don’t do everything we want; dueling me-too press releases (ERMs at the moment);
- Can we afford not to care? We need to move more nimbly; Service Oriented Architectures for Dummies is coming out! (you know you’ve hit the bigtime when there’s a Dummies book on the topic)

The alternative?
- Orchestrated disintegration: flexible ecosystem of business processes that come together as needed and as defined by us!

What might a library SOA look like?
- Oriented towards services
- As you look at the ecosystem of processes, there’s a consistency to it
- What are the business processes of a traditional ILS? Discovering content, describing content, borrowing physical item, getting new content (acquisitions), etc.
- traditional and non-traditional services for discovering content: search for known item; browse related items; make recommendations; read reviews; browse related items using relevance-ranked filters (long tail stuff); human-mediated description (cataloguing!); our peers are using automated description services (amazon’s statistically improbable phrases and capitalized phrases; shape recognition and colour maps, audio (flickr, etsy, pandora, etc.);

The wifi crapped out on me at this point so I lost all Peter’s great closing notes about who else is talking about SOA in libraries. Thankfully, he’s got his slides online here so check them out for more info.

- ILS has it’s roots back to 1948, but it came together in the 60s.
- the ILS has gotten quite complex, but the ecosystem hasn’t evolved with it
- we’re an edge market, we don’t get a lot of attention – not a lot of hands working on building us good systems
- at the same time, user expectations are at an all time high
- what makes something good at inventory control, does not make it a good UI
- if you look at enterprise systems, they tend to be plug and play
- Art wrote an article in 2001 called The End of the Integrated Library System – didn’t happen then
- In the book Guns, Germs & Steel, the author argues that technologies never work the first time
- now there are so many high quality open source products and the ecosystem has caught up with the software
- there are hybrid models – e.g. very few vendors recognize that they don’t need to build their own web servers or databases – they get them from someone else. There are more open source components creeping into the system – great possibility to mix & match.

How does IT work now?
- we don’t have a lot of great metrics
- there’s a lot of tinkering and a lot is based on broad statements
- it’s elusive, changes constantly

2 New Building Blocks that might fit into the ILS
- we think of the web server and relational databases as part of the ILS at the moment

ERP system (enterprise resource planning) – JD Edwards, PeopleSoft, etc.
- considered some of the most complex systems built
- late to the open source game – only 5 out there
- Ofbiz: an open source ERP.
- just making its way through the Apache incubator process (apache is the gold standard for open source projects)
- this could be a profound building block for us

Lucene
- state of the art indexing
- open source toolkit
- this could be the basis of a common index format
- could be a piece of the ILS landscape soon

Where should we be at the end of today? From the audience:
- A manifesto!
- We’ve been pretty vocal about our unhappiness with the ILS: where’s the action after this? a working group? a discussion list? how do we take this forward? should we create something new?
- Practical hooks for interoperability with other systems
- A way to keep the discussion going and build collaborations and partnerships
- Some discussion on where do we go with a practical approach. What practical advice can we give to our vendors?
- Engage the skeptics on the issue of open source options (skepticism has been the default position on an open source ILS).
- How do we make the people who deal with the patrons and the user experience partners in this?
- How do we set a direction for the future given that We Are Not Happy with what we’ve got?! (amen!)
- When we pay to have things done, how can we make sure that other people/libraries can also benefit from that?

Beth worked with Ranganathan’s five principles to frame her talk (which provided great context, but I only got notes on four of them! Bad, bad conblogger). She also worked in some examples of work that’s being done on the Oakville Public Library’s website redesign (which is not live yet, although we saw a couple of teaser shots and it looks excellent!) and results of surveys and interviews done with OPL’s patrons. The survey and interview questions peppered throughout the notes below refer to this.

————

Two questions:
- is there room for user-created content in OPACs?
- is it worth the risk (the risk of both doing it and not doing it)?

1. Books are for use.
- patrons depend on browsing for serendipity
- one interview question was: while in the library how do you usually discover new materials? Browsing was #1, looking at stuff on the return cart was #3, asking the librarian was #7. People are already collaborating in a way — it’s the notion that if someone else checked this out, it must be worthwhile! (Ed note: we saw a screenshot of OPL’s new site that is going to capture this in the web environment, but I won’t give anything away! Trust me, it’s cool!)
- the opac the way it exists now is like closed stacks – you have to know the author or title you’re looking for!

Interview question: if you could provide one piece of advice for your library, what would it be?
- improve the website. A patron noted that they start their search in amazon, see what people like, and then go to the OPAC!

Interview question: do you ever read reviews ratings and recommendations from other users?
- almost 75% said yes

Consider: what do patron’s library collections look like?
- library thing
- delicious monster
- bookcrossing
- building community around collections

2.Every reader his book.
- how do we find the right book for the right person at the right time?
- the democratic principle, every citizen his/her opinion
- how do we allow and make connections between different voices? Libraries have an interest in stewarding this where commercial providers don’t.
- amazon.com’s listmania: peer sharing of opinions
- it’s really about trying to find ways to allow those individual voices to connect.

3. every book its reader
- an idea of exposing the collection, ensuring that everything gets read.
- it’s not about taste elevation or being cultural gatekeepers.
- if we give them possibilities, they will choose possibilities.
- serving the long tail – but the long tail might concentrate demand through less and less choices.
- too many choices? We need “curators”. That’s what amazon.com does so well.
- interviews at OPL – searching a topic in the OPAC got way less than an amazon.com search, and the survey participants liked that. They felt that the library was curating for them.
- Cyber-balkanization: how do we think about this tradeoff vs. community?
- the Netflix business model is very much like libraries – ways of presenting people with choices and options.
- bestseller trap – reinforcing and perpetuating the cycle. We’re not changing it in any way.
- “the modern librarian is only happy when his readers make his shelves constantly empty”
- discovery: cross-referencing and showcasing – not just new, but old treasures too.
- building connections: per Ranganathan, it was all about subject classification. Maybe that doesn’t really work anymore.

Interview question: showed survey participants amazon.com tags and 50% of them thought tagging a book in amazon.com meant adding it to your wish list or shopping cart. If you look at the tags, they really aren’t meaningful. E.g.: harry potter is tagged with “harry potter”. Is this meaningful?
- it’s still early days for tags, we need to explore this in the long term. Tags need to be:
- evaluative: numeric, semantic
- descriptive: aboutness, offness
- associative: also recommended

4. Save the time of the reader.
- Jakob Nielsen (usability guru) says users spend most of their time not on your site, but on other sites. So figure out how to make your site like others!
- radical trust and smart system design go together.
- if you build it will they come?

Wrap-up:
- the only way it would work (i.e.: user-created content in OPACs) would be for library systems to work on it together. The only way to get critical mass, especially on books and resources that aren’t mainstream bestsellers, is if you get lots of users creating that content (user-reviews, etc.).
- user-created content management systems are far less forgiving of the need for constant ‘redesign”.
- bottom line? We all need to do this together!