Virtual Reference Realities
Current Research
and Customer Feedback
Karen Hyman & Peter Bromberg
South Jersey Regional Library Cooperative
Public Library Association Conference
February 26, 2004
QandANJ … what is it?
An LSTA-funded project staffed by 45 libraries of
all types in the NJ Library Network.
Free, interactive, search assistance on the Web.
Available 24/7.
Live since October 1, 2001.
Wildly popular with library customers of all ages.
QandANJ…
Why are we doing it?
Market libraries and library services
Meet needs and expectations of 21st
century customer
Immediacy
Shock value
Go where the people are
Collecting Customer Feedback
We’ve used two surveys:
1. Pop-upsurvey captures immediate
reaction
2. Follow-upsurvey filled out demographic
picture, quality of experience and user
behavior.
Collecting Customer Feedback
High response rate to popup feedback form
Quantity and Quality of data
• Over 10,000 responses to pop-up, “reaction”
survey
• 250+ responses to follow-up survey
• We have satisfaction and demographic
information AND thousands of free-text
comments.
Keeping it simple!
• Radio buttons for quick
answers
• Comment boxes for more
detail
4 point, color-coded, scale.
Optional email for follow-
up survey
Privacy statement
What are they telling us?
Age Percent
18 and under 45.8%
19 – 35 17.5%
36 – 59 27.4%
60 and up 9.3%
What are they telling us?
Grade Percent
College 17.2%
High School 20.7%
Middle School 23.5%
Elem. School 5.2%
Professional 33.4%
What are they telling us?
Was your question answered in a
reasonable amount of time?
Yes 73.7%
No 14.0%
Could be quicker 12.4%
Our pop-up survey respondents were...
Often students
More female
Younger
Satisfied with service
Likely to use again
Follow-up Survey
We wanted more detail on customer
demographics and satisfaction.
And to correlate with:
• Library behaviors: past and future
• Opinion of libraries
• Demographics
691 Customers provided emails
251 Responded
Rating of Service: a solid B+
Overall Service: 3.62
Quality of Help: 3.69
Consistency: 3.55
Courteousness: 3.82
Convenience: 3.84
Ease of Use: 3.79
No tech. problems: 3.56
Accuracy: 3.46
Follow-up Survey
Demographics
Female/male ratio (67/33)
Respondents tended to be older
(only 13% under 18)
Ethnically diverse by heritage and languages
spoken in the home (just like N.J.)
Follow-up Survey
Fairly educated (41% had attained more
than a 4 year degree)
Computer literate.
• 97% have a pc in their home
• 88% use for word processing
• 41% use for banking
• 67% use for shopping
• 43% used other chat-based services
Follow-up Survey
Mostly Library Users, but not all...
12% visit library yearly or less often
4.2% NEVER seek help from a library
when they have an information need.
Follow-up Survey
Why do they visit the library?
75%: Check out materials
73 %: Research/Find Information
59%: Browse
40%: Get assistance from librarian
25%: Study
23%: Use library computers
18%: Attend library programs
10%: Attend non-library programs
Follow-up Survey
QandANJ : How do customers find us?
35%: Link on library webpage
27%: Word of Mouth (friends, family, teacher)
13%: Article in paper or magazine
13%: Search Engine
9%: Posters/Bookmarks
7%: Link on non-library webpage
2%: Television
"How has your experience with QandANJ
affected your opinion of libraries?
Improved: 66%
Not affected: 32.5%
Worsened: 1.5%
QandANJ …
Real feedback from real users
QandANJ… Real feedback from real users
Interaction with a live person: “I was talking to a real
person not a computer!”
Interaction with a nice person: “I liked that someone
came on and didn’t make fun of me or anything about
my question.”
Expertise of the librarian: “The librarian found
information that my Mom and I had been looking for
and could not find no matter where we looked.”
Always open: “I got the answers I needed, especially
when the library was closed.” “[My] first contact was at
2 am Sunday morning.”
QandANJ… Real feedback from real users
The library came to me: “I am house-bound …this
opportunity is greatly appreciated.” “I didn’t have to
go out in the rain.”
Fast: “I like the fact that you get an immediate
answer to your question instead of having to click
on a hundred different links and still not find the
information you need.”
Awesome: “I am continually astounded by this
service and it’s results never cease to please.”
In other words…
“The speed of getting precise information. The
fact that it was free. The polite
professionalism of the librarian. The fact that
it is valuable information. The fact that you
can access it 24 hours a day. The fact that
libraries are keeping up with the future of
accessing information and sharing that with
the public. It is quite a wonderful, helpful
thing.”
Conclusions and surprises
Our follow up survey people were-- middle
aged, educated, computer savvy, library
users -- they exist!
QandANJ users LOVE the real-time human
connection, available 24/7
People still need help finding answers.
Taking live reference service to the web 24/7
has had a demonstrable positive impact on
peoples’ perceptions of libraries!