This session will review significant enhancements to Pipeline Pilot in recent releases. Areas covered are: Professional client, administration, security, integration, databases, and collections (chemistry, next gen sequencing, documents and text, statistics, and imaging).

1. (ATS3-PLAT01) Recent developments
in Pipeline Pilot
Ton van Daelen
Sr. Director, Platform Product Management
ton.vandaelen@accelrys.com

2. The information on the roadmap and future software development efforts are
intended to outline general product direction and should not be relied on in making
a purchasing decision.

3. Introduction
• Target audience
– Administrators
– Developers
• What will you learn?
– What is new that affects me or my users (and me indirectly)
• More of What’s New in
– (ATS3-PLAT07) Pipeline Pilot Tips, Tricks and Challenges
• Slides and examples will be made available

4. Topics
• Release history
• Administration
– Performance enhancements
– Server monitoring
– Usage monitoring
– Protocol validation
– Security
• Integration
– .NET components
– Python (prototype)
• SharePoint integration

5. Pipeline Pilot Release History
• Pipeline Pilot 8.0 – Dec 2010
– Design mode 1-4
– Local subprotocols
– Data flow monitoring 5.0
6.0
• Pipeline Pilot 8.5 – Aug 2011
– Support growing deployments 7.0
– Isentris and Notebook integration 8.0
8.5
– Support for building ‘apps’
• Pipeline Pilot 8.5 CU1 – Nov 2011
– Protocol comparison
9.0
– Performance

6. Science and Applications Release History
Imaging (HCS)
Reporting (Canvas elements)
Advanced data modeling
Chemistry (MDL formats)
Reporting (Stylesheets)
7.5 Imaging (DICOM)
Direct cartridge
Database Search
Imaging (Big Image Explorer)
Chemistry (Content, NEMA, Cheshire)
8.0 Next generation sequencing
8.0
Isentris and Notebook integration
ChemReg
Collections BioReg
Apps
8.5

7. Mini-Survey
• What server version(s) are you running?
– Pre-7.5?
– 7.5
– 8.0
– 8.5

8. Administration

9. Performance Testing
• Performance increasingly critical because of ‘apps’
– Both Accelrys and custom
• Performance testing
– Track our progress
– Spot errors
• How do we do this?

10. Performance Testing

11. Threads / Windows

12. Throughput / Windows

13. Linux / Threads and Throughput

14. Pipeline Pilot Configurations Compared (ODBC Test Case)
Latency of PP 7.5, 8.0 & 8.0 PoolID Latency of PP 8.0 & 8.0 PoolID
20000 1200
18000
16000 1000
Avg Response Time (ms)
Avg Response Time (ms)
14000
800
12000
10000
600
8000
6000 ODBC latency is 44% Improved scalability
400
lower than 7.5 and under load
4000
22% lower than 8.0
2000 w/out pooled servers 200
0
1 5 10 20 50 100
0
ODBC 7.5 235 202 192 218 1558 18622 1 5 10 20 50 100
ODBC 8.0 224 166 164 156 482 1015 ODBC 8.0 224 166 164 156 482 1015
ODBC 8.0 (PoolID) 122 137 117 122 246 367 ODBC 8.0 (PoolID) 122 137 117 122 246 367

15. Server Maintenance and Monitoring
• Administration portal
– Run diagnostics
– View and cancel running
jobs
– View log files
– Monitor CPU and memory
• Pro client
– Server status indicator

16. Server Maintenance and Monitoring
• External tools
Windows Performance Counters Ganglia (Linux)

17. Usage Monitoring
• Why?
– Internal accounting (‘Which dept is using imaging?’)
– Justify investment (‘Are we using reporting?’)
– Administrative (‘What protocols haven’t been used in 1 yr?’)
• Approaches
– Usage report in admin portal
– Custom analysis of log files
– Protocol database analysis using Component Reader
• More in: (ATS3-PLAT11) Advanced Planning

18. Admin Portal Usage Report

19. Usage Report Example
Introduced in 8.5. For pre 8.5 servers, install protocols from
ftp://download.accelrys.com/protocols/SummaryReport.zip.

20. Logfile Folder
Installation logfiles.
Remote logfiles. Allows for
manual copying if server is
not connected Local logfiles.

21. Usage Log Analysis Through Protocols
• Usage protocol in C:Program
FilesAccelrysPPSappsscitegicadmindatarootxml
C:Program FilesAccelrysPPSlogsusagePipeline Pilot.log

22. Analysis Example
Runs by User Runs by ClientType Most Frequent Protocols

23. Analyzing the XMLDB
• Questions to ask
– Non-empty Password parameters?
– Empty required parameter field?
– File references?
– Shortcuts as copies?
– Protocol for a particular user?
– Recent protocols?
– Components of a certain collection?
– Deprecated components?
• Combined with log file
– What components were used recently?
– For a migration, what protocols were run recently?

24. Analyzing the Xmldb
Protocol info:
• Author
• Date
• Component ID
Component info:
• Data source references
• Passwords

25. Component Parameters
ParamName ParamValue ParamType
Source dataImagesCardsREADME.txt URLType
Maximum LongType
SourceTag None StringType
Keep Properties PropertyPreviewType
HasColumnNames TRUE BoolType
Delimiter FigureOut StringType
MultipleValueDelimiter None StringType
ParseNDimensionalArrayStrings TRUE BoolType
Gotcha! Additional Options
CommentLines
GroupType
StringType
UndefinedString StringType
Component Reader Example.xml Accelrys or custom AllLines FALSE BoolType
TrimWhitespace TRUE BoolType
component? CheckSyntax TRUE BoolType
AddEmptyProperties FALSE BoolType
OnGeneralError Halt StringType
OnDataTypeError Halt Date registered
StringType
ComponentAttributes ComponentReturnsPass StringType
Registrant scitegic/generic StringType
RegistrationDate Sat Feb 26 16:31:03 2011 StringType
ComponentDisplayName Read README.txt StringType
DerivedFrom Delimited Reader StringType
CustomErrorText Caption StringType
ComponentIcon DocumentNode StringType
ComponentDisabled 0 LongType
ComponentGUID {7D2AF7C7-A077-4201-B10B-8E8283A5EADD} StringType
ComponentPoint 90 -95 StringType
ComponentHelp 100 StringType
Component Sticky Notes
RegistrationVersion 8.4.5.18
Help text StringType
StringType
ComponentLocalID 0 LongType
Component Path ComponentsData Access and ManipulationFile ReadersDelimited Text R
StringType

26. Component Parameter Interface

27. Component Parameter Interface

28. Protocol Validation
• Inspect protocols before moving to production
• Rules checking:
– Hard-coded passwords
– Bad file of database references
– Broken shortcuts
• Configurable for different deployment modes
– Standard protocol, Reports, Web services, RSS
Feeds, SharePoint, etc.

29. Protocol Validation
• Validation can be set up as hard requirement for publication
• Operation is cancelled on validation errors
*Not a standard feature. Request from support
(ATS3-PLAT03) Enterprise Protocol & Package Deployment

30. Automating Component Best Practices
• Component Review-o-Matic
Protocol
– Checks Documentation
– Checks Parameter Names
• Create your own custom rules
See: ComponentsDatabase and Application
IntegrationUtilitiesPrototypesUsabilityComponent Review-o-Matic

31. Security

32. Windows Integrated Authentication
• Run Web Port without providing
username /password
• Supported via Kerberos and NTLM
• Requirements
– DOMAIN authentication
– Kerberos only works on remote
servers. Negotiation on local servers
uses NTLM instead of Kerberos, so
some restrictions may apply
– Pipeline Pilot Apache service must
have valid Service Provider Name on
the DOMAIN server. Running as "Local
System" usually fulfills this
requirement
– Configuration required on server and
client’s browser

33. Windows Integrated Authentication
• Requires Active Directory settings
• Enables Secure Services
• Authentication provided by NTLM/Kerberos
• Limitations
– Delegation not supported
– Accessing network resources (e.g., proxy server authentication with
credentials, UNC paths)
– Using PP credentials for components that require authentication
(e.g., database, readers, integration, etc.)
– Running protocols from within a protocol (e.g., RunProtocol
PilotScript function, parallel subprotocols)

34. Server and Client Configuration
• Run Apache service as a user:
– To set up Kerberos service principal name for the Pipeline Pilot server, run the following
on the Domain controller:
• setspn.exe -a http/SERVERNAME DOMAINapache_username
– On the Pipeline Pilot server, use the Services tool and set the Logon User for the
"scitegic_apache_8_5_0" service.
• Client Configuration
– Internet Explorer:
• Add the server as a trusted site (Tools > Internet Options > Security > Trusted Sites > Custom Level >
User Authentication > Logon).
• Select Automatic logon with current user name and password.
• If your server is already part of the Local Intranet, select Automatic logon only in Intranet zone.
– Firefox:
• Browse to "about:config" and add the server names to the following preferences:
– network.negotiate-auth.trusted-uris
– network.automatic-ntlm-auth.trusted-uris

35. Pass-through (Impersonation)
Pipeline Pilot Web Services
Webform Basic
Scientist
User Token
Database
SciServer
ODBC
Web Services
Basic

36. Integration

37. Pipeline Pilot Integration Points
Web Browser
Run Protocol .NET Client Java Client SOAP Client
Professional Client Command Line Web Port JavaScript Client
Client
.NET Client
JavaScript Client SDK Java Client SDK Web Services API
SDK
Pipeline Pilot Enterprise Server
Web Apps Web Services API Admin Portal Help Portal
Protocol Runtime Environment (scisvr)
VB Script VB Script
Run SOAP & Telnet / ODBC /
(On (On Java Perl Python .NET SSH / SCP
Program HTTP FTP JDBC
Client) Server)
Java Perl .NET
VB Script Cmd Line
Classes Scripts Classes
REST /
SOAP
HTTP Cmd Line DBs
Service
Service

38. Component Development Update
• .NET On Server
– Points to a compiled .NET Assembly
• Dynamic .NET
– Embed your .NET code directly into
the Server Component
• Leverage .NET supported
languages
– C#
– VB.NET
– IronPython

39. A look at building a .NET Server Component

40. .NET Components Demo
• Hello world using dynamic C#
• Hello world using compiled C#

41. Python (Prototype)
• Requires
– ActiveState Python
– .NET runtime
• Windows only
• Prototype now available
– Looking for testers
properties = component_datarecord.GetRoot().GetProperties()
queryString = properties.GetByName(u"queryString").GetValue().GetString();
params = urllib.urlencode({'corpID': queryString})
f = urllib.urlopen("http://vm-discoweb-dev.corp.mpi.com/rest/Compound/lookupCorpID/getCompoundProperties?%s" % params)
properties.Define("Response", f.read());

42. SharePoint

43. Pipeline Pilot & SharePoint
Web Services Run Job Web Parts (3)
Pipeline Pilot SharePoint Bridge
User Created
IT Created
Read/Write Document Library
ISV Created
Protocols Lists
Pipeline Pilot SharePoint

44. PP Scientifically Enables SharePoint

45. Deploying Custom Web Parts
• Problem/Need: Want to create Pipeline Pilot-based Web Parts and deploy to
organization
• Solution: Create packaged (“standalone”) Web Parts (Chapter 4, SharePoint
Bridge Admin Guide)

46. Using Custom Web Parts
• Custom Web Parts are
available in Web Part Gallery
• Provides a much simpler
mechanism for end-users

47. SharePoint Document Library
• Read from / write to
document library
– Use standard Reader/Writer
components
– With Domain
Authentication, R/W will
simply work
– Alternatives
• Copy between document
repository and temporary
local version
• Add authentication options to
Reader/Writer/Copy
components

48. Summary
• Reasons to upgrade
– Administrative
– Development
– Science and applications
• Plan your upgrade
– Engage Accelrys support and services
– (ATS3-PLAT11) Advanced Planning
• Resources
– Component development guide
– Accelrys Community: http://community.accelrys.com/
• Other sessions of interest
– (ATS03-PLAT07) Pipeline Pilot Tips, Tricks and Challenges

49. The information on the roadmap and future software development efforts are
intended to outline general product direction and should not be relied on in making
a purchasing decision.
For more information on the Accelrys Tech Summits and other IT & Developer
information, please visit:
https://community.accelrys.com/groups/it-dev