IBM Connect 2017: Your Data In the Major Leagues: A Practical Guide to REST Services
1. Your Data In the Major Leagues:
A Practical Guide to REST Services
Serdar Basegmez
Managing Director,
Developi Information Systems
DEV-1383
IBM Connect 2017 Conference, 20-23 February 2017
2. • IBM Champion (2011 - 2017)
• Developi Information Systems, Istanbul
• OpenNTF / LUGTR / LotusNotus.com
• Featured on…
• Engage UG, IBM Connect, ICON UK, NotesIn9…
Serdar Başeğmez
3. Agenda
• RESTful Architecture
• Practical Implications
• Advantages of RESTful Services
• Providing REST Services for IBM Domino
• Consuming REST Services for IBM Domino
• Wrap-up
4. RESTful Web Services
Representational state transfer (REST) is an architectural style used for
web development. Systems and sites designed using this style aim for
fast performance, reliability and the ability to scale (to grow and easily
support extra users). To achieve these goals, developers work with
reusable components that can be managed and updated without
affecting the system as a whole while it is running.
Source: https://en.wikipedia.org/wiki/Representational_state_transfer
6. Old School Web Applications
Source: https://speakerdeck.com/jeffschenck/rest-easy-api-security-done-right
User Interface Business Logic Datastore
Front-end Back-end
ASP, PHP, CGI, Web Agents, JSP, etc.
← HTML, CSS, JavaScript
Forms →
7. Web Applications Evolving
User Interface Business Logic Datastore
Front-end Back-end
Async web apps, Ruby on Rails, Django, JSF, XPages, etc.
← HTML, CSS, JavaScript
Forms, AJAX →
8. Web Applications Evolving
User Interface Business Logic Datastore
Front-end Back-end
Modern Web frameworks, Angular.js, React.js, etc.
← HTML, CSS, JavaScript
← REST →
9. [Web] Applications Evolving
User Interface Business Logic Datastore
Mobile Applications
Back-end
Modern Web frameworks, Angular.js, React.js, etc.
← HTML, CSS, JavaScript
← REST →
Front-end
10. [Web] Applications Evolving
User Interface Business Logic Datastore
Mobile Applications Back-end
Modern Web frameworks, Angular.js, React.js, etc.
← HTML, CSS, JavaScript
← REST →
Front-end Microservice Microservice Microservice
12. Stateless / Cacheable / Layered
Every request processed independently
Client does not care who cooked the meal in the kitchen
Everything cacheable
⇣
Scalable, Robust, Resilient
13. The Conversation Makes Sense!
Source: http://www.bizcoder.com/a-fresh-coat-of-rest-paint-on-a-soap-stack
15. The Conversation Makes Sense!
http://appserver.company.com/apps/contacts.nsf/
GiveMeTheContactWeNeedPleaseAgent?OpenAgent&id=1522
or…
http://appserver.company.com/api/contacts/1522
16. Conventions on URLs
GET http://appserver.company.com/api/contacts
GET http://appserver.company.com/api/contacts/UK/London
POST http://appserver.company.com/api/contacts
Retrieve Contacts / Create a new Contact…
17. Conventions on URLs
GET http://appserver.company.com/api/contacts/1522
PUT http://appserver.company.com/api/contacts/1522
DELETE http://appserver.company.com/api/contacts/1522
Retrieve/Update/Delete the Contact resource with id=1522…
18. URI GET PUT POST DELETE
/contacts/ List Contacts Replace Contacts Create New Contact Delete Contacts
/contacts/id Retrieve a Contact Replace a Contact N/A (generally) Delete a Contact
Source: https://en.wikipedia.org/wiki/Representational_state_transfer
Conventions on URLs
19. Unconventional uses in URLs
GET https://api.twitter.com/1.1/statuses/show.json?id=1234567890
Retrieve the Tweet with id=1234567890…
21. Some Inspiration
• Alternative Front-ends
• Alternative Web Frameworks (Angular.js, React.js, etc.) / Mobile Applications
• Integration with Custom applications/services to share data and process
• Getting a list of customers from CRM, let SCM to initiate Purchase Request on Domino
• Upload a file into Box, using tone analyser from IBM Watson, send message into Slack
• Webhooks
• Notify or get notified about what happened
• Other
• Enhancing In-page user experience with grids, quick searches, better type-aheads, etc.
22. Example: Slack Bot for XSnippets
Slack Server
Slack Client
OpenNTF Server
XSnippets.nsf
XSlack REST
Provider
/xsnippets [params]
Message
Response
GET Request FT Search
Search Results
Message Response
GET http://openntf.org/xsnippets.nsf/slack.xsp?text=xagent
Request:
Here are search results for 'xagent'
1. Scheduled XAgent - NON SSL <http://...>
2. Use @Transform to build JSON and consume the output in an XAgent <http://...>
3. Controlling the HTTP response status code in XPages <http://...>
4. Build JSON from NotesView with support of multi-value fields <http://...>
5. Scheduled XAgent -SSL Encrypted <http://...>
Response:
23. Example: Box Webhooks Integration
Box Servers
Box Web
Company Server
Demo.nsf
REST Service
POST Request Actions
200 Success Response
POST http://MyServer.com/MyRestServiceAddress
{
"source": {
"name": "someFile.xyz",
"parent": {
"name": "someFolder",
"type": "folder"
},
......
},
"trigger": "FILE.UPLOADED",
"type": "webhook_event",
........
}
Request:
BoxSync
Upload File
Delete File
Comment
24. RESTful Services on Domino
IBM Domino Server
Web Apps Agents
IBM Notes Client
RESTful
Services
As Consumer
Browser
Remote
Applications
As Provider
Provider
Consumer
26. Domino Access Services
• Provided and Supported by IBM
• Fully functional RESTful API for Domino Data
• Access to Views/Folders/Documents/Collections/Fields/Calendar
• Support for caching
• Mail and FreeBusy services added by ExtLib OpenNTF version
• Inherently uses standard security model (ACL, readers/authors…)
• Server-level/Database-level/Design-level control over availability
27. Domino Access Services
• No coding needed, Immediately available after a few settings.
Enable on Server Enable For Database Enable For Views
28. Domino Access Services
• Drawbacks:
• No control over the data!
• Reading a document —> Read All fields
• Creating a document —> No checkpoints!
• No place for business logic!
• What about actions? The responsibility is on the consumer
• Everything or Nothing
• Exposes internals
• You should trust consumers and the environment…
29. REST Components (ExtLib)
• Provided and Supported by IBM
• Access to Views/Folders/Documents/Collections/Fields/Calendar
• Customizable component for RESTful access
• Computed/Filtered columns, Custom search, etc.
• Event model helps building business logic on top of REST model
• Custom REST Service
• Write your own SSJS or Java bean
• Write your CSJS routines for async access (Remote Service / JSON-RPC)
• Dojo support for single page model
30. REST Components (ExtLib)
• Setup REST component(s) on your page.
• Minimal coding, no administrator needed.
Add to your XPage Add a Service Configure Options
31. REST Components (ExtLib)
• Drawbacks:
• Careful with the code organization…
• You might inevitably end up with a spaghetti code!
• Error handling is crucial
• Prefer CustomRestService with a Java bean for more advanced stuff…
• Not optimised for performance and scalability
• Difficult to follow RESTful URL Convention
• e.g. https://someserver.domain.com/database.nsf/somepage.xsp/service/…
32. Hardcoding (Web agents, XAgents, Servlets…)
• Old school way to create services, but still quite useful for some cases.
• Great if you have pre-existing code (e.g. Lotusscript libraries, etc.)
• Customizable, flexible and simple way to create any service
33. Hardcoding (Web agents, XAgents, Servlets…)
• Drawbacks:
• Hardcode everything…
• e.g. Header/parameter extraction
• Careful with the code organization…
• You might inevitably end up with a spaghetti code!
• Error handling is crucial
• Difficult to follow RESTful URL Convention
• e.g. https://someserver.domain.com/database.nsf/xagent.xsp?…
• e.g. https://someserver.domain.com/database.nsf/someagent?OpenAgent&…
34. Apache Wink Servlets
• IBM Domino includes Apache Wink 1.1.2
• An implementation of JAX-RS
• JAX-RS: ‘Java-ish’ way to define RESTful services
• Create JAX-RS based REST services on top of OSGi plugins.
• Complete Java solution, extensible with custom providers
• Compatible with OpenNTF Domino API
• Code reusability outside IBM Domino world.
36. @Path("/contacts")
public class ContactResource {
private DominoAccessor accessor = new DominoAccessor(ContextInfo.getUserSession());
@GET()
public Response getContactList( @QueryParam("start") int start, @QueryParam("count") int count) {
List<Contact> contactList = accessor.pullContacts(start, count);
String result = ModelUtils.toJson(contactList).toString();
return Response.ok(result, MediaType.APPLICATION_JSON).build();
}
@Path("/{id}")
@GET()
public Response getContact(@PathParam("id") String id) {
Contact contact = accessor.findContact(id);
if(null == contact) {
throw new WebApplicationException(Response.Status.NOT_FOUND);
} else {
String result = ModelUtils.toJson(contact).toString();
return Response.ok(result, MediaType.APPLICATION_JSON).build();
}
}
}
{
"zip": "13202",
"state": "NY",
"lastName": "Abbate",
"middle": "J",
"country": "US",
"emailAddress": "Jessica.J.Abbate@trashymail.com",
"number": "DLEY-ACLH6Y",
"city": "Syracuse",
"firstName": "Jessica"
}
Contact Resource Class
Contact Resource
Short JSON Representation
37. Apache Wink Servlets
• Drawbacks:
• Plugin only
• Difficult if you are not familiar
• Takes time to learn
• Overkill?
• Not suitable for small projects and simple needs
• Apache Wink is old school
• Not that bad, IBM still using Wink. But Apache took another way.
• Alternatives: RESTEasy, Jersey, Apache CXF, etc.
• Integrating a new module into Domino might be an issue
38. Providing RESTful Services on Domino
Benefits Challenges Suggested When?
Domino Access Services
(DAS)
No Backend Code
Zero-setup
Limited Control
No Business Logic
Exposes the Internals
Simple internal integrations
ExtLib Components
for REST
Less Backend Code
Minimal Setup
Partial/Full Customization
Error Handling
Spaghetti Code Risk
URL Convention
Simple needs for a limited
scope
Hardcoding
(XAgents, Web agents,
Servlets)
Tailor-made
(Almost) No Learning Curve
Hardcoding Everything
Spaghetti Code Risk
URL Conventions
Very specific needs for a
limited scope
Apache Wink Servlets
Tailor-made
Based on JAX-RS
OSGi Benefits
Learning Curve
Barrier to Entry
Large scope
implementation,
API Design
40. How to Consume any RESTful Service?
• It is just an HTTP Request…
• Questions to ask:
• Lotusscript or Java or SSJS?
• On-demand or Background?
• The remote service has an SDK?
• Who is going to be authenticated?
41. Java or SSJS or Lotusscript?
• Lotusscript
• No internal support for Networking…
• On Windows platform, we can use COM objects
• Call Java via LS2J
• Server-side JavaScript
• Use Java!
• Java
• Core Java provides URLConnection class for basic operations
• Open source libraries are preferred (e.g. Apache HttpComponents)
42. Connecting LotusScript
• Pulling exchange rates (When needed or on a schedule)
Set rates = CreateObject("msxml2.DOMDocument" )
rates.async = False
rates.validateOnParse = False
rates.setProperty "ServerHTTPRequest" ,True
If rates.load("http://www.tcmb.gov.tr/kurlar/today.xml" ) Then
Set currencies =rates.getElementsByTagName("Currency" )
For i=0 To currencies.length - 1
Set node=currencies.item(i)
If node.attributes.getNamedItem("Kod").value="USD" Then
For j=0 To node.childnodes.length-1
If node.childnodes.item(j).NodeName="ForexBuying" Then
forexBuyingUSD = node.childnodes.item(j).nodeTypedValue
Elseif node.childnodes.item(j).NodeName="ForexSelling" Then
forexSellingUSD = node.childnodes.item(j).nodeTypedValue
End If
Next
‘ Elseif for others…
End If
Next
GetExchangeRates=True
Else
GetExchangeRates=False
End If
Using OLE Automation
(or LS2J alternatively)
XML Parsing
Windows only!
43. Connecting Java
• Pulling exchange rates via Java (When needed or on a schedule)
public Double receiveEurUsdRate() throws Exception {
CloseableHttpClient httpclient = HttpClients.createDefault();
HttpGet httpGet = new HttpGet("http://api.fixer.io/latest?base=USD");
CloseableHttpResponse response = httpclient.execute(httpGet);
try {
HttpEntity entity = response.getEntity();
// This is the worst practice ever!
// We don't check anything! Everything can go wrong...
JsonJavaObject ratesMap = (JsonJavaObject) JsonParser.fromJson(JsonJavaFactory.instanceEx,
EntityUtils.toString(entity));
JsonJavaObject rates = ratesMap.getAsObject("rates");
// We can write values into a NotesDocument
return rates.getAsDouble("EUR");
} finally {
response.close();
}
}
Create an HTTP client
Get a response
Convert to JSON
* Networking is a restricted operation for Agents!
Agent Security should be configured.
44. Connecting SSJS
• Pulling exchange rates on an XPage action
<xp:button
id="button1" value="What is the exchange rate?">
<xp:eventHandler
event="onclick"
submit="true"
refreshMode="partial"
refreshId="computedField1">
<xp:this.action><![CDATA[#{javascript:
viewScope.EurUsdRate = test.ExchangeRate.receiveEurUsdRate()
}]]></xp:this.action>
</xp:eventHandler>
</xp:button>
We can also call Java!
* This code will run multiple times for each page cycle.
Always cache values!
45. A Simple Example
• Pulling exchange rates (When needed or on a schedule)
• Running from an Java Agent
• Apache HttpComponents and IBM Commons should be imported.
• Agent Security should be configured
• Running from an XPage / Java or SSJS
• Apache HttpComponents should be imported.
• Time-outs and service hiccups should be considered
• Values should always be cached!
• Lotusscript
• Platform specific vs. LS2J
46. When/How to Connect?
• Web Application triggers the RESTful conversation.
IBM Domino Server
Web Apps
RESTful Service
Web Client
1. Click/Save/Update
2. REST Request
3. REST Response
4. Render Response
Latency!!!
47. When/How to Connect?
• Web Application performs the RESTful conversation at the browser.
IBM Domino Server
Web Apps
RESTful ServiceWeb Client
1. CSJS Action 2. AJAX Request
3. AJAX Response
4. Process Response
Latency!!!
CORS (Cross Origin Resource Sharing)?
48. When/How to Connect?
• Notes Client performs the RESTful conversation
IBM Domino Server
Agent Manager
RESTful ServiceNotes Client
1. Run local code (Java/LS) 2. REST Request
3. REST Response
Latency!!!
4. Process Results
49. When/How to Connect?
• Notes Client performs the RESTful conversation via a Server Agent
IBM Domino Server
Agent Manager
RESTful Service
Notes Client
1. Agent.RunOnServer()
2. REST Request
3. REST Response
4. Read Results
Latency!!!
Latency!!!
50. When/How to Connect?
• RESTful conversation happens in the background
IBM Domino Server
Agent Manager
RESTful Service
1. Agent or DOTS run on a schedule
2. REST Request
3. REST Response
4. Process Results
DOTS
51. When/How to Connect?
• More complicated scenarios…
IBM Domino Server
REST Service
RESTful Service
RESTful Consumer
1. REST Call
3. REST Request
4. REST Response
6. Return response
Latency!!!
2.Process the Request
5. Process More
52. Why not SDK?
• Many remote services provide a Java SDK or Wrapper
• Wrapper vs SDK
• Wrapper: Well-defined data model (e.g. POJO classes)
• SDK: No need to deal with low level operations
• Generally supported by the provider or the community
• But…
• Compatibility issues (especially on Java Agents)
• e.g. IBM Watson SDK requires Java 7
• JVM Security issues
• Too much dependency, Poor documentation, etc.
53. Authentication
• Who is going to be authenticated and how?
• Predefined credential or application key
• Single Sign-on
• Users with their own credentials on remote (OAuth, Saved password…)
54. Authentication
• Predefined credential or application key
• Server to Server authentication
• Every REST request should be configured with proper keys
• Credential or Application key should be secured
• Not suitable for CSJS Model
55. Authentication
• Users with their own credentials on the remote system
• Single Sign-on
• Valid for integrations when SSO is available (e.g. IBM Connections)
• Works within the same domain!
• The remote system expects a token, generally in cookie form
• CSJS model will work without any modification
• When HTTP request passes through the server, cookie should be transferred
56. Authentication
• Saved Password:
• Easy for the developer, but not convenient!
• Security risk, changing passwords, two-factor authentication, etc.
• Many applications do not allow credentials via API
• OAuth
• Users can delegate their authentication to your application for a limited scope
• Domino has no native support, but possible with some effort (See Demo)
• IBM Social Business Toolkit SDK brings the support for OAUTH (well, sort of…)
57. Consuming RESTful Services
Usage Examples
XPages
Java - SSJS
Through SSJS or Java beans
Called when needed
Sending message to Slack
Integrations to IBM Watson
Social Media Interaction
Agents
Java - Lotusscript
Scheduled agents
Can be called by Notes Client
Periodically pulling exchange rates
Pushing data to remote service
Training IBM Watson AI
Using SDKs
XPages - OSGi
Access to services using external libraries
Called when needed
Scheduled using DOTS
Accessing IBM Connections
Integration to Box
60. Browser
Box-Domino 3-legged OAuth Dance
Web Application
Domino Server
0.ClientId is needed for initiation
https://Box_Api_Url?client_id=<App_Client_Id>&redirect_uri=<Rest_Service_Url>
We will build a Link to initiate the flow…
TokenStore
Callback REST
61. Browser
Box-Domino 3-legged OAuth Dance
Box Servers
Web Application
Domino Server
Login Page
0.ClientId is needed for initiation
1.Redirect User to the Login Page
TokenStore
Callback REST
62. Browser
Box-Domino 3-legged OAuth Dance
Box Servers
Web Application
Domino Server
Login Page
0.ClientId is needed for initiation
1.Redirect User to the Login Page
TokenStore
Callback REST
Box will ask the user
to login and request
permission for the
application.
63. Browser
Box-Domino 3-legged OAuth Dance
Box Servers
Web Application
Domino Server
Login Page
0.ClientId is needed for initiation
1.Redirect User to the Login Page
2.User Accepted the Permission Request…
TokenStore
Callback REST
64. Browser
Box-Domino 3-legged OAuth Dance
Box Servers
Web Application
Domino Server
TokenStore
Callback REST
Login Page
0.ClientId is needed for initiation
1.Redirect User to the Login Page
3.Box Issues a GET Request with
a temporary Authentication Code
GET
2.User Accepted the Permission Request…
65. Browser
Box-Domino 3-legged OAuth Dance
Box Servers
Web Application
Domino Server
TokenStore
Callback REST
Login Page
0.ClientId is needed for initiation
1.Redirect User to the Login Page
3.Box Issues a GET Request with
a temporary Authentication Code
4.We Issue a POST Request with
ClientSecret+Authentication Code
GET
2.User Accepted the Permission Request…
POST
66. Browser
Box-Domino 3-legged OAuth Dance
Box Servers
Web Application
Domino Server
TokenStore
Callback REST
Login Page
0.ClientId is needed for initiation
1.Redirect User to the Login Page
3.Box Issues a GET Request with
a temporary Authentication Code
4.We Issue a POST Request with
ClientSecret+Authentication Code
5.Box responds with tokens
AccessToken+RefreshToken
GET
2.User Accepted the Permission Request…
POST
RESPONSE
67. Browser
Box-Domino 3-legged OAuth Dance
Box Servers
Web Application
Domino Server
TokenStore
Callback REST
6.Save Tokens
7.DONE!
Login Page
0.ClientId is needed for initiation
1.Redirect User to the Login Page
4.Box Issues a GET Request with
a temporary Authentication Code
5.We Issue a POST Request with
ClientSecret+Authentication Code
6.Box responds with tokens
AccessToken+RefreshToken
GET
2.User Accepted the Permission Request…
POST
RESPONSE
Now, we can act on
behalf of the user…
68. Some Tweaks
• SSL Issue
• TLS 1.2 is required in most cases.
• Technote: https://www-01.ibm.com/support/docview.wss?uid=swg21985289
• HTTP Proxy/Gateway
• Proxy through the Domino Server for CSJS requests (Mostly for security reasons)
• Options
• IBM Domino provides HTTP-proxy servlet
• Programmatically create a gateway using “Remote Service” component
• Testing
• REST service: Postman (Chrome Plugin / Standalone)
• REST consumer: MockBin RequestBin httpbin
71. Takeaway
Download and play with demos
Experiment simple services
Get yourself familiar with RESTful Services
Download OpenNTF projects
Study on RESTful design practices
Have a Pet Project
73. Notices and
disclaimers
continued
Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other
publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of
performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should
be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such
third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR
IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
PARTICULAR PURPOSE.
The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents,
copyrights, trademarks or other intellectual property right.
IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise
Document Management System™, FASP®, FileNet®, Global Business Services ®, Global Technology Services ®, IBM
ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®,
Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®,
PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®,
SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli®, Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®,
X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions
worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is
available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.
74. Resources
• Serdar Başeğmez: Demo for this session
https://github.com/sbasegmez/IC17RestDemo
• Serdar Başeğmez: Apache Wink Template and Demo
https://github.com/sbasegmez/RestAssuredDemo
• Serdar Başeğmez: Box - SBT Demo
https://github.com/sbasegmez/Blogged/tree/master/CloudFile
• Graham Acres / Serdar Başeğmez: The Journey to Becoming a Social Application Developer (IBM Connect 2014)
https://speakerdeck.com/sbasegmez/bp308-the-journey-to-becoming-a-social-application-developer
• Stephan H. Wissel: Custom REST service in XPages using a service bean
https://wissel.net/blog/d6plinks/SHWL-9Q55QL
• John Dalsgaard: REST Services in IBM Domino/XWork
https://www.dalsgaard-data.eu/blog/rest-services-in-ibm-dominoxwork
75. Resources (cont.)
• Eric McCormick: Series on JSON Data with Java in XPages
https://edm00se.io/json-with-java-in-xpages
• Thomas Ladehoff: REST Services with the XPages Extension Library
https://www.assono.de/blog/d6plinks/REST-Services-with-the-XPages-Extension-Library
• Paul Withers: XPages OSGi Plugins series
http://www.intec.co.uk/xpages-osgi-plugins-1-an-introduction/
• John Cooper: Domino OSGI (Part 1) - Configuring Eclipse for XPages OSGI Plugins
http://developmentblog.johnmcooper.co.uk/2014/05/configuring-eclipse-for-xpages-osgi-plugins-part1.html
• Toby Samples: JAX-RS or THE way to do REST in Domino series
https://tobysamples.wordpress.com/2015/04/28/jax-rs-or-the-way-to-do-rest-in-domino-part-1/
• Jesse Gallagher: Eclipse Tutorial for Domino Developers
https://github.com/jesse-gallagher/eclipse-tutorial-oct2015/wiki/Java