The document contains the 11 commandments for administrators and developers working with Lotus Notes and Domino. It discusses important principles for each role, such as change control, security practices, writing maintainable code, understanding versions, and monitoring systems. The speakers provide guidance and warnings about common problems that can occur when these best practices are not followed.

2. BP306
The 11 commandments..
Paul Mooney| Senior Architect | Bluewave Technology
Bill Buchan | Director | HADSL
®

3. I
(State your name)

4. Pledge solemnly to a flag that I have affection
for....

5. Or to a flag that I like in some particular
way....

6. Or possibly to a flag that I never even heard
of.....

7. To fill out the evaluation for this session

8. And to fill out the evaluation in full

9. And to include fantastic comments on the
quality of the speakers

10. And also, to complete the following

12. •Add LocalDomainServers to my terminated group

13. •Add LocalDomainServers to my terminated group
•Remove Option Declare

14. •Add LocalDomainServers to my terminated group
•Remove Option Declare
•Delete 10 user accounts at random

15. •Add LocalDomainServers to my terminated group
•Remove Option Declare
•Delete 10 user accounts at random
•Move all code to initialised

16. Because we are all individuals...

17. So say we all :)

18. BP306
The 11 commandments..
Paul Mooney| Senior Architect | Bluewave Technology
Bill Buchan | Director | HADSL
®

19. Agenda
We shall cover:
 What do you need to know
 What MUST you do
This is a technology free area
 This is a discussion on the methodology of being
– A good Administrator
– A good Developer

20. Administrators and Developers
Are different types of people
Some would say evil versus good
 Or good versus evil
Consider it heaven versus hell
 But, how is that best described?

21. Administrators and Developers
Are different types of people
Some would say evil versus good
 Or good versus evil
Consider it heaven versus hell
 But, how is that best described?

22. Time to get into character
Developers

23. Time to get into character
Administrators

24. The format
We have 11 commandments each
 11 for Administrators
 11 for Developers
All have a common goal
 Defend your services
Your service is key
 servers and applications
 everything else is secondary

25. Admins 1: What makes a good admin?
What makes a good Admin:
 Stubborn refusal to change crap
 Absolute hatred of everyone
 Anti-social behaviour and general air of Menace
 Feet up on desk - means everything is running
 Favourite word: No
Because
 Change is bad
 Service is good
– Are you meeting your SLA?
– If not - what’s the cause?
– Can you address this with low risk?

26. Developers 1: What makes a good Developer?
What makes a good developer
 Good Attitude, empathy
 Ability to change complex systems without failure
 Generally good code ability
 Multiple code platform strengths
 Very good customer facing skills:
– Don’t call the customers ‘idiots’. Ever
 Communication
 Favourite word: Yes
Because
 Change is Good!

27. Admins 2: Thou shall have change control!
Why?
 Prevention of infrastructure changes on the fly keeps your service stable
– Protect your service
 Change control is risk management.....
Problems caused?
 “one change” has caused massive service failures over the years
– Point release upgrade without research...
– Slight change to replication/routing
How?
 Implement strict change control in
your domino environment.
 What’s your process?
 Where is your documentation?

28. Developers 2: Thou shall have change control!
Why?
 Change control is risk management.....
– All changes are well understood, documented
 Change Management protects the environment and protects YOU
Problems caused?
 “one change” has caused massive service failures over the years
– Want to generate 4m documents per HOUR in 56 replicas worldwide, creating
56gb of replication conflicts ?
How?
 Three stage environment
– Development Sandbox
– User Test Environment
– Production

29. Admins 3: Thou shall understand security!
Why ?
 Failure to secure means you’re a target
Problems Caused?
 Anonymous Editor access to directory from Internet
 Everyone’s ID still in the NAB with password LotusNotes
 Anonymous Editor access to donation site - all credit card
details visible
 Open ECLs?
How?
 Be aware of the Lotus Domino Security Model
 Conduct informal audits
– Catalog.nsf, ‘ACL view’.

30. Admins 3: Thou shall understand security!
7 layer security is based on:
 Cross Certification
 Deny/Allow Server Access Lists
 Database ACL
 Reader Fields / Form / View security
 Encrypted Fields
 Client execution
Know your server security page
 Anonymous access switched on ?
Know your group infrastructure and how this relates to security
Who is allowed Full Access Administration ? Only you ?
Check your certificates.
Should they be rolled over now ?
 64-bit RSA is now factorable.
 Upgraded your certificates since v6 ?

31. Developers 3: Thou shall understand security!
Why?
 Failure to understand security leads to:
– Private information becoming public
– Unhappy Customers
– Reputation as an ass
Problems Caused?
 Your users should only be allowed to see what you allow
them to see.

32. Developers 3: Thou shall understand security!
How:
 Developers have a multi-tier security model - Within
applications
– Database ACL, Roles
– Reader/Access Fields
– Public. Private encryption keys
 Understand Reader Fields
– Multi-value array of CANONICALISED Names
– Use the notesItem.isReader or notesItem.isAuthor field
– Always have a fallback reader field
– What happens when AdminP removes your last reader?
 Understand Encryption:
 Use the ones most appropriate for the application

33. Admins 4: Thou shall deploy the ECL
Why?
 Execution Control Lists is a 10 year old Lotus Notes (client) feature
 Using it, you can control the execution of code on clients
– Prevent unsigned code execution
– Provide extra security
Problems caused
 Domino is virus free?
– Too many times we have witnessed any code available to execute in Domino
– A good/bad developer could have a fun time there

34. Admins 4: Thou shall deploy the ECL (cont)
How?
 Simple Steps
 Register two signer id files
– One for restricted execution
– One of unrestricted execution
 NEVER give them to developers
 Grant these ids rights to run code in the server documents

35. Admins 4: Thou shall deploy the ECL (cont)
How?
 Create an administration ECL from the admin client
 Add in the signer id files with rights
 Use policies in directory to push out the ECL to the clients
– Replace/Refresh?
Be aware
 This is NOT as simple as it is documented
 In sites with little control
 All code must be reviewed
 This is a slow process!

36. Developers 4: Thou shall write maintainable code
Why?
 In Lotus Notes, we tend to change applications far more than
in other environments
 We also tend to release code far more frequently - our RAD
roots mean more versions
 This means that the majority of our time is not spent
writing new shiny code - its updating the existing stuff
 If updating the existing stuff is hard, then it costs more time
and money
Problems Caused:
 1200 lines of code in a single function, no errorhandling
 Printing out a subroutine (onto many pages of paper) and
drawing lines between if/then/else code sequences
 Buggy, ‘scary’ code

37. Developers 4: Thou shall write maintainable code
How:
 Split up routines into less than a screenful
 Use Errorhandling
– On every function thats more than 2 lines long.
– Call a central errorhandler that gives you ALL your detail
 Use defensive coding:
– ‘Trust No-One’
– Check everything at the start of the function
– Bale out if anything goes wrong
 Re-use code
– So that the majority of your application has already been
tested!
 Write Business Logic in Script libraries, OO code
– So that they can be applied to multiple UI’s.
●
Notes, Web, Web Service
Test early, test often

38. Admins 5: Thou shall understand your environment
Why?
 Domino administrators are typically not SAN administrators
 Domino administrators sometimes are not involved in hardware acquisition
Problems caused
 Slow/poorly managed RAID arrays SERIOUSLY hamper server performance
– Split arrays for services
– Transaction logs
– View index buildings
– Data
– DAOS
– OS
 Sharing arrays on SANS with other products can
make performance totally inconsistent and
unreliable

39. Developers 5: Thou shall measure twice, cut once!
Why?
 More time thinking, less time coding, leads to better
architecture
 You can always write bad code on a good architecture, and
it’ll work. Even writing good code on a bad architecture is
always a disaster
Problems Caused:
 Ever had a project that just never finished. Always more and
more bugs?
 Ever had a piece of code that EVERYONE has had to fix?
 Fragile, Difficult to extend applications
 Always fighting fires, never writing code?

40. Developers 5: Thou shall measure twice, cut once!
How?
 Spend more time listening and less time talking
 Get hard and non-ambiguous specification, and get them in writing
 Write specification for your application - even in the ‘About’ document:
– It forces you to think how the application will work
– Anyone maintaining the application has a fighting chance
 Show the customer your changes, and make sure they understand the impact
 Get the customer involved in testing so they understand what they get
 When making changes, always figure out what NEEDS to change first
 Dont have a ‘rush of code to the fingers’

41. Admins 6: Thou shall review the catalog
Why?
 The catalog.nsf database records a LOT of information about all databases in your
environment
– Even ones that are not listed as “include in database catalog”
– Open the ($replica) view for proof
●
Or build a custom view
Problems caused
 A neglected catalog.nsf leads to poor security
 Applications end up with default access to high
– Or worse, anonymous access
 Go to the ACL view and look at -Default-

42. Admins 6: Thou shall review the catalog
How
 Periodically review the catalog.nsf database
 Make it part of your monthly review
 Use the DDM to monitor ACL changes
– Security review probes

43. Developers 6: Thou shall understand @Formula!
Why?
 So you don’t end up writing something horrible -
unmaintainable, buggy, slow - in the wrong language
 You should use the best features of each language
Problems caused:
 Ever seen XML parsing in raw LotusScript?
 Ever seen a Lotusscript language parser written in
LotusScript ?
 Ever seen Bubblesort implemented as @Formula Language -
and then copied/pasted between 50+ fields ?
 It causes unmaintainable, horrible, code.

44. Developers 6: Thou shall understand @Formula!
How
 Developers - Good developers - should understand:
– LotusScript - the key to large, maintainable, predictable
code in Notes Client and Scheduled Agents
– @Formula - Form and View specification, error handling,
lookups
– Java - Good for network access, Web Services, etc.
 You can call an agent from another agent - and pass a string
as a parameter
– Allows you to call Java from LotusScript or vice versa
– Pass a NoteID of a document as a parameter
– use the Notes document to pass values between agents

45. Admins 7: Thou shall monitor mail usage
Why?
 Domino is amazingly resilient
– Router task is outstanding for managing volume/size of messages
 HUGE PROBLEM - ITS TOO GOOD
– Often ignored
– Quotas exceeded - throw more disk space at them
Problems caused
 Unrestricted mail usage internally can hamper server performance
 Typically external mail size is managed, but often we are too forgiving on internal
messages
 Attachment control
 Quota control
– Wait for Paul’s rant on Quota control

46. Admins 7: Thou shall monitor mail usage
How
 Message size restrictions
 Rules - More efficient in a way, but difficult to manage
 Quotas
– Implement quotas!
– Push back at the business, protect the service
 Ask Lotus to change the way attachment quotas work?
 Implement DAOS!

47. User Stupidity : Space saving
Detailed, scientific study taken....
Attachment Duplicates Understand “anykey”
42

48. User Stupidity : Space saving
Detailed, scientific study taken....
Attachment Duplicates Understand “anykey”
42

49. User Stupidity : Space saving
Detailed, scientific study taken....
Attachment Duplicates Understand “anykey”
42

50. Developers 7: Thou Shall Understand Versions!
Why?
 If you use a version of notes to develop code that is ahead of
the version that your clients and your servers use, you may
write some code that will be impossible for the client or the
server to execute
Problems Caused?
 You use features that will not run on your users Notes Client
 Web services - you save an R8 Web Service that does not run
on your R7 Servers
 Forms, hide-whens, pages, views, all stop working...
 Servers crash

51. Developers 7: Thou Shall Understand Versions!
How ?
 You should always be developing in a version that is not
greater than your oldest client version or your oldest server
version
 Good practice to have multiple versions of designer, to make
sure that you can test and develop in the right place
– Difficult to do with nd8. VMWare is your friend.

52. Admins 8: Thou shall maintain your servers
Why?
 Wrong maintenance tasks running on wrong servers
 Wrong parameters
 Too frequent/not frequent enough!
Problems caused
 Tasks running causing DBID’s to change
– Invalidating transaction log backups
 Tasks running into daytime
 Tasks clashing

53. Admins 8: Thou shall maintain your servers
How?
 Understand the tasks!
 What does compact do?
 Update versus Updall
 Fixup.. when.. consequences
http://www-01.ibm.com/support/docview.wss?uid=swg27006573

54. Developers 8: Thou shall keep up with Evolution!
Why?
 Things change - new releases, new techniques, changes to
business
 New versions mean new capabilities
Problems caused:
 Old, ugly inefficient apps
 Old, ugly inefficient code
 Old, ugly inefficient developers

55. Developers 8: Thou shall keep up with Evolution!
How?
 Attitude:
– You never will know it all
– Everyone knows something you don’t
– Get used to it, and learn from others
 Learning:
– Courses, Books, Blogs,
●
http://www.planetlotus.org
– Skim the blogs - always something useful
 Resources
– http://openntf.org - a repository of free, open templates
and code
 Conferences and User Groups
– Lotusphere, iLug, UKLug, SNUG, etc.
– Notes.net
Remember: You are not alone!

56. Admins 9: Thou shall NOT set 300 year lifespans
Why?
 Its done because its easy
 No need to re-certify like you had to before
 No chance of hundreds of people expiring at the same time
Problems caused
 Security issues
 Are your deny lists correct?
 Domino is as secure as you make it

57. Admins 9: Thou shall NOT set 300 year lifespans
How?
 Use your registration settings in policy documents
 Use the expiration view in the nab/catalog
 License tracking?
 Delegate user management to help desk
– SECURELY

58. Developers 9: Thou Shall Log!
Why?
 Domino has a multi-platform, server and client based, event
triggered and scheduled agent execution manager in THREE
different languages
 Code is executed on servers (on multiple platforms) in
multiple locations and on multiple versions
 Lets not pretend ALL code is perfect ALL the time on ALL
platforms in ALL circumstances.
– That’s what Project Mangers think
 We need to know before the users tell us:
– What went wrong, Why, Where and When
– How we can fix it
 We need to know when agents have NOT ran
– Yearly scheduled agent anyone ?

59. Developers 9: Thou Shall Log!
Problems Caused:
 Complex Business Applications stop, or are unreliable
How:
 You need to add Error Handling to all your scheduled routines
– Java, LotusScript, @Formula
 You need to collect these logs in a logical manner
 You need to ensure that the process of logging doesn’t slow the application
down significantly, nor generate terrabytes of useless information
– Just like an unmonitored DomLog.nsf
– We have CEO’s for that
 Check out OpenLog on OpenNTF
– By Julian Robichaux - openNsf.com

60. Admins 10: Thou shall manage the minions
Why?
 You owe a level of service to your end users... yes... I said that
 They need to leave you alone
 You need to make that happen
Problems caused
 Look at your support calls
– They are resolved sometimes in firefight mode
 Look at them again..
– How can you prevent the problem from happening
 Top 10 support calls are the most frequent
– Wrong location document
– Wrong replica
– Archive issues etc etc

61. Admins 10: Thou shall manage the minions
How?
 POLICIES
– Lock them down
– Manage archives.. use server archiving if you can
 Location documents
– Delete all the location documents that are not used
●
Home dialup?
– Especially if you are on citrix
 Bookmarks
– Use policies
●
Dont give them an excuse to say
they can’t find a database

62. Developer 10: Thou Shall Be Organised!
Why?
 We are asked to do lots of things each day
– Develop complex systems
– Fix this bug
– Add this feature
– Send me the specifications for...
Problems Caused:
 You forget stuff
 You’re always apologising
 You never have enough time to do it all

63. Developer 10: Thou Shall Be Organised!
How?
 Track all bugs for all systems in a database
– Check out OpenNtf for bug-tracking templates
 Track all your personal actions
– Get Things Done - session with Eric Mac and Dave Allen
 Stay on top of things
 Allocate time in your day for actual management of
information as opposed to work
 Always add ‘contingency’ to your estimates
 Less Haste More Speed
 Always ask yourself: Is there a smarter way to do this?

64. Admins 11: Thou shall learn NSDs
Why?
 The NSD is a great tool with excellent data
 Unfortunately it is written in Kilngon
 It IS trying to tell you what the problem is...
– You just have to give them a chance
Problems caused
 Repetitive crashes
– Could be easily preventable
 Poor service
 Disrupted users
 Homicidal bosses

65. Admins 11: Thou shall learn NSDs
How?
 Take a look at last years “Die Hard” presentation
– Introduction to NSDs
 Download and use the NSD debugger tool from IBM sandbox
– http://www.pmooney.net/pmooney/pmooneynet.nsf/d6plinks/PMOY-77DR4R
 The NSD is trying to tell you what’s wrong
– Typically a little practice and you end up liking them!

66. Developers 11: Thou Shall Learn Web Services
Why
– Web Services are an application to application protocol to
exchange information
– Web Services give a huge amount of integration potential
to your applications
Problems Solved:
– No more using LEI/NotesPump/Notrix to mirror other data
sources in Notes
– Real time, online lookup of other data
– Far more integration on the .glass

67. Developers 11: Thou Shall Learn Web Services
How?
 Notes 7.x allowed Domino to host Web services
 Notes 8.x allows notes clients to consume web services
– But alternatives available for older releases
 Notes 7 allows LotusScript based web services
– So there’s no excuse!
 nd8 allows Java and LotusScript based web services
 Use SoapUI for testing
– http://www.soapui.org
 Other platforms can easy consume Domino Data
– BlackBerry Handsets using MDS Studio
– .Net applications using Visual Studio 2005 or later
– Adobe Flex
– And many many more!

68. Wrapup
Admins and Developers share more goals than we all suspect:
 Service Stability
 Environment Stability
 Predictable change management
 Easy life
Professionalism
 Choose where/when to have excitement thrust upon you
 Dull, in this context, is professional!
– Chinese Curse: May you live in interesting times

69. Thank You
Paul Mooney
 pmooney@pmooney.net
 www.pmooney.net
Bill Buchan
 bill@hadsl.com
 www.billbuhan.com

70. Legal disclaimer
© IBM Corporation 2008. All Rights Reserved.
The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is
provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice. IBM
shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect
of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.
References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in
this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way.
Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results.
Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary
depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed.
Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International Business Machines Corporation in the United
States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both. the Simpsons is a trademark of Fox TV, in the United States, other countries, or
both.
Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.
Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.
Batteries not included. Objects in the mirror may appear closer. Shinks when wet. Buyer beware. Investments may go up as well as down.
Portions (c) Ned Flanders, Smithfield. Magnetic North may vary. Without Prejudice.