Presentation & notes from "Measuring & Improving Library Value" seminar on 29 November 2012 in Sydney.

1. Supporting researchers &
research publication:
impact measured by
more than just metrics
@malbooth

2. Supporting Researchers
The main problem of impact research is,
that influences on an individual are
manifold and that therefore it is difficult
to trace changes and improvements
back to the library.
Roswitha Poll, Philip Payne, (2006),"Impact measures for
libraries and information services", Library Hi
Tech, Vol. 24 Iss: 4 pp. 547 - 562
http://dx.doi.org/10.1108/07378830610715419
Other problems (quoted from ref above):
. A service can have different value and outcome for different user groups. A training session in special databases will have less effect
on freshmen than on postgraduates who need these resources directly for their work.
. Data that could be relevant for demonstrating impact are not available because of data protection rules (e.g. individual data about
grades in exams).
. The data or correlations found in projects until now are in most cases not comparable, as differing methods were used.
Standardization of methods will be necessary to allow for benchmarking of results.
. Long-term effects can often not be assessed if the users are no more available for tests or surveys.
. All methods that have been tested until now are time-consuming.

3. Supporting Researchers
Academics are seen as "producers", whose research
is expected to focus on topics of commercial value
and whose "output" is measured against a single
scale and graded like sacks of wheat. The universities
themselves are encouraged to teach and research not
what they think is intrinsically worthwhile but what is
likely to be financially most profitable. Instead of
regarding each other as allies in a common enterprise,
they are forced to become commercial competitors.
Sir Keith Thomas, Council for the Defence of British Universities
& a fellow of All Souls College, Times Higher Education, 8
November 2012
http://www.timeshighereducation.co.uk/story.asp?storycode=421722
I think it is worthwhile keeping this sentiment in mind as we start delving into measuring things, with the competition that is
implicit in such processes.
Read the whole article if you can as it is a worthy reminder about the true purpose of universities and how it is being distorted by
bean counters and their obsession with compliance, KPIs, rankings and putting measures on mostly intangible assets.
Ah, I feel better now ...

4. Supporting Researchers
ENVIRONMENTAL SCAN
global competition for
research standing |
research funding
uncertainties |
research publications costs |
trend to “Gold” open access
(OA) models |
UTS research strategy,
themes & Framework for
Doctoral Education |
Universitätsbibliothek Humboldt-Universität zu Berlin
Most of these points will be familiar to everyone. We are constantly reminded of the importance of our standing in various global
university rankings. I know that it does actually matter for reputation and income and I’m not arguing against that, just the amount
of importance it is given over other important roles and achievements in universities.
Politically and economically in Australia the funding for research from government sources looks at best to be uncertain for the next
3-5 years. If we look elsewhere, will that have an impact on the objectives (or focus) of that research, just as government funding as
done?
There is an alarming trend towards Gold OA, not just in the UK. Academic publishers are mostly global, so it will inevitably have
even more of an impact in Australia than it is already having. We need to come up with strategies to deal with it so that early career
researchers (at least) are not disadvantaged by their inability to raise research funds. We need to also make sure that we don’t
unrealistically assume a funding obligation that will easily blow library budgets out of the water.
At UTS we enjoy a very clear research strategy (next slide), themes (or research strengths) and a new framework for doctoral
education. This gives us very clear directions that we can aim at and keep aligned to.

5. Building a Leading Research Performance
http://www.research.uts.edu.au/docs/mcu-research-strategic-plan.pdf
Supporting Researchers
Just an online snapshot, more can be seen at the link above.

6. Supporting Researchers
STRENGTHS
networks, partners |
diversity of expertise |
OA: repository & publishing |
clear strategy, targets,
framework |
trusted & enjoy institution
wide respect |
copyright/IP expertise |
data management &
curation |
Philological Library of the Free University of Berlin
So apologies for using the SWOT format, but it just allows me a logical way to organise some talking points that cover my thoughts
on this subject.
STRENGTHS:
Libraries have well developed networks for all kinds of things. In many ways we are excellent networkers and we make good partners
for researchers.
We usually have a good deal of diverse expertise and that is certainly the case at UTS. Many of our librarians have excellent
reputations for their expertise in all of our faculties and in most research centres.
At UTS Library we run the OA research repository and we run UTSePress (the largest publisher of OA journals in Australia). That
gives us good credibility in this space, gets us partnering with many academics for their research and publication and keeps our
professional skills sharp in relevant contemporary fields. I think it is a huge advantage.
As I said we have a very clear UTS research strategy, targets and framework. It is easy for us to see where we could have the best
impact and help the most.
At UTS we are also trusted and enjoy institution-wide respect. We also occupy a “neutral” position within the university and are not
seen as a competitor by anyone else. That is a strength to exploit.
We hold the UTS Copyright Officer within the Library and provide much advice to researchers and others about protecting their IP
and not signing it all away when publishing.
As mentioned earlier we manage the UTS data repository and two large research data archives, so we have hands-on experience at
data curation.

7. Supporting Researchers
WEAKNESSES
invisibility | own
perceptions | OA is
misunderstood |
limited budget |
inflexibility | risk
averse | cautious |
slow | lacking
expertise (in some
fields) |
article processing
charges (Gold OA) | Faking It, The Metropolitan Museum of Art
our language |
Some of these weaknesses are our own perceptions and others are how others may see us.
In many ways we are still at least partially invisible. I think we need a more visible profile in order to stay relevant, engaged and
aligned with UTS priorities and directions and in order to be seen as valuable partners.
Sometimes our own perceptions of ourselves are a little too negative and we probably need to be more confident about the services
we offer and our own professionalism.
Open Access is still misunderstood, so we still need to raise awareness about its benefits, be better advocates for it, and teach people
about it to correct some false assumptions.
Our budgets are certainly limited. Sometimes worthy initiatives can only be afforded at the expense of something else.
In some respects we still need to learn to be more flexible and be prepared to compromise. We should be less defensive of our
position and also some of our “traditions”.
I think we need a better sense of urgency, we need to learn about managing, not avoiding risk and thus become less cautious in our
approach. Sometimes we are far too slow to respond or to initiate action.
In some fields of research we will always lack professionally trained expertise, so we must learn to make a valued contribution in
other ways.
We currently say no to APCs, but it is probably an inflexible position that we cannot hold for much longer without damaging our
reputation and the university’s research standing. I think we will have to find a compromise position and a better way forward. It
isn’t a black and white issue.
The language we use can sometimes be confusing and unhelpful to our clients. We need to use more plain English and terms that any
of our clients can clearly understand.

8. Supporting Researchers OPPORTUNITIES
OA promotion | data advisory |
connecting | personal
relationships | bibliometrics |
research metadata collection
& harvesting | online help | lit
review | profile creation &
management | alignment |
multi-disciplinary research |
specific software help |
social networks | OA
Brooklyn & Manhattan Bridges
publishing innovation |
crowd-funding? | altmetrics
This is a recent image I took (from the Empire State building) of the bridges to Manhattan from Brooklyn. There is also a tunnel
under the river in much the same area. I think libraries are more like bridges than a destination. Bridges imply opportunities.
We must do more to promote Open Access, not just raising awareness, but educating our researchers and academics about it and
addressing the many myths. We have to help make it work, to improve the way it can be used and to assist however else we can.
Providing a professional data advisory service will just become more and more important into the future. That means helping with
everything to do with data - across the whole curatorial process: finding, using, creating, managing (describing, arranging, preserving,
etc.) and sharing.
One of the best things librarians can do is to help with connections - connecting researchers to each other, expert assistance in the
library, to data archives, to sources of knowledge, to inspiration, etc.
The development of trusted personal relationships with our researchers is really the only way to make all of this work. If we simply
rely on ads on our website or email we are fooling ourselves. We have to get out on campus and into the faculties, research centres
and schools to get into the faces of our researchers and market our valuable series or they will simply dry on the vine.
We are already working with researchers with regard to bibliometrics and helping them to maximise the impact of their research
publications. It is in some ways related to the collection and (potentially) automated harvesting of research metadata (for ERA and
HERDC purposes). We have for some time been looking at the Symplectic product that is used by some unis (e.g. UNSW) to
harvest metadata & citations, manage identities, etc. This has been done with our university research office and a proposal is going
forward to implement that at UTS. It should also assist with profile creation and management.
We have online help available via our website (see links in later slides), but it is nowhere near enough by itself. Researchers in
particular need the personal help.
I know it is time consuming, but some researchers do need our professional assistance with literature review and I know that some
our librarians spend time on this for certain researchers.
Our efforts much be aligned with the university’s key research goals and priority targets to have any impact. Our resources are
limited, but as I said earlier the goals in UTS are very clear for us.
At UTS there is a growing trend in multi-disciplinary research and we need to be able to respond to this appropriately. If we are too
siloed in our own arrangements we will be of no use. Learning how this works for researchers and how best to support such
initiatives is now a real need.
In a later slide you will see examples of the ways we assist with specific software and other online skills. These are particularly
targeted to the needs of our researchers and those classes are well attended and the subject of ongoing requests for more.
Some of the most popular awareness raising that we’ve done is to make our researchers aware of various, sometime discipline
specific social networks that exist and can be very useful to them.
Maybe libraries can do more in a creative sense in supporting format changes and even multi0media in OA publishing. We are more
agile and have the resources to do that, perhaps ahead of traditional publishers.
The competition for research funds is now very fierce, but perhaps in some areas that have a high public profile or that address
particular needs crowd-funding is a possibility. We will look into assisting with this in potential areas.

9. Supporting Researchers
http://altmetrics.org/manifesto/
See also:
http://www.swets.com/blog/altmetrics-for-librarians-and-institutions-part-i
http://www.swets.com/blog/altmetrics-for-librarians-and-institutions-part-ii
http://www.swets.com/blog/altmetrics-for-librarians-and-institutions-part-
iii
If you’re unaware of this burgeoning field, check out the landing page and then have a quick scan of the three-part blog post linked
above.

10. Supporting Researchers
THREATS & CHALLENGES
loss of reputation, trust |
OA gold model |
becoming too digital |
permissions, control |
ignorance, avoidance |
time available to
researchers & work-life
balance |
invisibility of our services |
missing the boat |
The Highline, NYC
These are some of the things that can (and sometimes do) go wrong if we don’t get it all right. So we need to be prepared to limit the
damage and change things to prevent recurrence.
The Gold OA model is a dilemma for us at present. I think it needs some for of compromise because we are on a hiding to nothing if
we simply dig in and stand on our principles. Inevitably some researchers suffer and we may well lose institutional reputation. Until
there is a long term solution, libraries must be part of the solution and make a contribution. I don’t think we should try to carry the
entire burden and fund all “memberships” and Article Processing Charges, but we need to help.
I think we could become far too digital and this would alienate many researchers, more particularly in some fields than others. As
I’ve said before, the human, personal element is very important.
Requiring too many permissions and too much control will alienate researchers and probably prevent real innovation.
If we don’t promote or market our services, people and facilities well enough we do risk them being ignored or even avoided by
researchers.
Researchers at different stages of their careers have very different times of the day available for the kinds of services we offer. It
means we have to be flexible in our offerings and be prepared to offer them at times that suit the target groups, not ourselves.
Sometimes I think we are in danger of missing the boat if we don’t move quickly and express our interest and enthusiasm with
initiatives. This has not happened recently at UTS, but with our new “creative intelligence” strategy, it easily could have if we had not
decided to become involved at all levels from the outset.

11. Supporting Researchers
HOW CAN WE TELL?
attendance | surveys |
data, stats | KPIs
(consultations, repository
ranking, citations in collection)
| rankings |
citations | partnerships |
feedback, satisfaction |
invitations, requests |
increasing acceptance,
inclusion | recommendations |
collaboration | creative
The famous @MissSophieMac initiatives | OA metrics |
altmetrics |
Many of these indicators will be familiar to everyone.
The formation of lasting partnerships with researchers is most beneficial and rewarding, but also hard to quantify and report on.
We do get expressions of satisfaction via several feedback mechanisms that we’ve set up, but again it can be hard to plug that into
the traditional reporting structures.
Being invited or requested, accepted, recommended or included in programs or researcher education/development or for research
projects is very high praise and recognition for the value added by librarians. Being included as a research collaborator is even higher
and rarer praise, but it can and does happen. Having supervisors recommend their charges for our training and development
programs is also a great indicator of success, for their value.
We’ve recently assisted to make some creative initiatives a reality for UTS researchers, particularly in the field of multi-media
research publication, visual communication research and multi-disciplinary research into collaboratively creative innovation.
We can also use OA metrics and altmetrics in addition to traditional measures to illustrate the success of our initiatives and services
such as our repositories, data archives and publications.

12. Supporting Researchers
Some examples
from UTS
Library ...
The following slides are mostly based on screen shots from our website, but I need to stress that whilst the programs might be
linked or listed there, they are mostly delivered in person by living, breathing librarians, both in the library and out where the
researchers are in their schools, faculties, offices and research centres. Those online links point to many of our programs that ONLY
flourish because of the human element and many deep personal relationships we've established. We are very active at engaging with
them on their own turf and taking our services to them. You cannot simply think that your work with researchers is done by putting
it on your website or sending them an email.
I think that with most researchers it is almost a case of "if you build it (online) they probably won't come" - you need to do much
more than that.

13. Supporting Researchers http://www.lib.uts.edu.au/uts-publications
I guess “Publications” is one of the key OA platforms we have for researchers - our online theses, managing our OA research
repository, and managing UTSePress which is the largest OA publisher of journals in Australasia.

14. Assistance with finding, using, creating,
managing & sharing research data
Consultations with
Library staff
Data Management &
Visualisation
workshops
Info on data archives
Seeding the
http://www.lib.uts.edu.au/research/data-archives Commons
Supporting Researchers ADA & ATSIDA
This is going to be a growth area in future years for sure. We offer data management consultations with library staff who have been
involved in managing data archives and these are increasingly popular with UTS researchers in line with our data intensive university
strategy. Data Management workshops have been popular for some time and recently we’ve successfully added basic visualisation
workshops which are also very popular.
We provide information (including online) about various data archives available to researchers (see above) and we’ve been one of
many universities involved in the ANDS Seeding the Commons project which aims to create infrastructure collect and transform
metadata about data collections and then publish it to Research Data Australia.
We also manage two social sciences data archives: the Australian Data Archive http://www.ada.edu.au & Aboriginal and Torres Strait
Islander Data Archive http://atsida.edu.au/

15. Supporting Researchers
Online support
http://www.lib.uts.edu.au/research
Here you can see our various online support initiatives that are aimed specifically at researchers. A key area is the support we provide
to researchers in the form of IP and Copyright advice about how to manage the intellectual property properly.

16. Supporting Researchers
Training program
http://www.lib.uts.edu.au/help/tours-workshops/research-skills-
postgraduates-staff
We offer very targeted training programs, all delivered by professional library staff and these are aimed at specific needs that we
identified in concert with researchers and research supervisors to assist in their development. Where necessary we bring in expert
collaborators to assist us in delivery.

17. Supporting Researchers http://www.lib.uts.edu.au/news/6118/research-week-2012
Research Week is a highly concentrated one week program that brings librarians, researchers and others who support or manage
research at UTS together. We don’t deliver all of the sessions, but it is a library initiative and has seen some changes and
improvements over the last two years it has been run. We will run it again in 2013 and planning is already underway!

18. Supporting Researchers
Shut up and Write is an initiative that came out of Research Week. We also get professional experts in to assist us where needed and
it has proved both popular and beneficial with researchers. Programs like RW and SUW also assist us in raising the profile of the
library as an active supporter of research at UTS.

19. Supporting Researchers
More research help
http://www.lib.uts.edu.au/research/tools
The tools listed above are those we’ve found on the web and we list them with brief comment, centrally on our website for
researchers who may need something in those fields. You can also see the facility for researchers to vote them up/down.
On the right is a screen capture that illustrates some of the research-specific programs we are running in Late November/early
December 2012. Power sessions for researchers have been very popular for a couple of years and NVIVO is an analytical tool that
we train many researchers to use by bringing in professional assistance.

20. Supporting Researchers
SOME LESSONS
alignment & keeping in step |
timing is sometimes everything |
to be strict or more open? |
blogs must have a pulse |
personalisation & voice |
OA & APCs |
sometimes you must “kill your babies” |
catering for different needs |
going deeper, less frequent |
spontaneous gratitude |
if you build it (online), they probably
won't come |
So, finally some of the things we’ve learnt along the way:
Our programs must be aligned with strategic directions in UTS research or they risk being irrelevant, avoided and having little or no
impact.
They need to be delivered at times of the year and the day that are convenient to researchers or they will not show up. This has
happened.
Sometimes ECRs and PhDs students need structure and strict obligations in our programs, so openness and extreme flexibility is
not always best. This comes from them!
If you want to keep a blog, it must have a pulse. It has to be kept up-to-date or people will not read it and then they will
unsubscribe.
Particularly with researchers, the personal touch and a real name are more important than institutional voice.
I think some form of compromise is needed in our position re “Gold” OA and APCs and I’m now doing some work to make that
happen.
We can’t be wedded to great initiatives or long-term programs if they are unsuccessful. If they don’t work they must be changed,
refocussed or dropped. This can sometimes prove VERY challenging.
Researchers are not that generic in their needs so we need to carefully cater for different needs and try to meet as many of them as
possible. One size certainly does not fit all in the research world.
In some of our programs, feedback from researchers is that we are more effective going deeper into a subject and offering them less
frequently than offering more opportunities at a more shallow level.
We do get some great spontaneous and unsolicited feedback from researchers and it is always great to hear about or to read.
Sometimes they make all the effort really worthwhile for us.
I think that with researchers ut is important to remember that online and email simply isn’t any where near enough. I have used
screen captures on these slides that are only pointers to the services we are offering. In many cases these are not at all online or even
delivered within the library. Our liaison and information service librarians have been very active at developing relationships with
faculty & getting services to them. I only have to hint at an interest and those guys never fail to follow up on it. The website alone
just isn't good at getting right in the face if researchers, you need warm bodies for that.

21. Supporting Researchers
Further reading
Claire Creaser, Valérie Spezi (2012), “Working together: evolving value for academic libraries”, report
commissioned by SAGE, June 2012: https://libraryvalue.files.wordpress.com/2012/06/ndm-5709-lisu-
final-report_web.pdf
Hajjem, Chawki, Harnad, Stevan and Gingras, Yves (2005) Ten-Year Cross-Disciplinary Comparison of the
Growth of Open Access and How it Increases Research Citation Impact. IEEE Data Engineering Bulletin, 28,
(4), 39-47. http://eprints.soton.ac.uk/262906/
Swan, Alma (2010) The Open Access citation advantage: Studies and results to date. http://
eprints.soton.ac.uk/268516/2/Citation_advantage_paper.pdf
Joshua Gans (2012), What an academic article of the future should look like, Core Economics (blog), 22
November 2012: http://economics.com.au/?p=9495
Arthur Hendricks (2010),"Bloggership, or is publishing a blog scholarship? A survey of academic
librarians", Library Hi Tech, Vol. 28 Iss: 3 pp. 470 - 477: http://dx.doi.org/10.1108/07378831011076701

22. THANK YOU