Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
From Bad Puns to Implementation
1. FROM BAD PUNS TO
IMPLEMENTATION
Daily struggles. Sock puppet jokes. Successful deployment.
2. Eric Arellano is a Systems Administrator with AppointmentPlus
based out of Scottsdale, AZ. Puppet and automation enthusiast.
Project owner of a successful Puppet deployment for an
enterprise level SaaS scheduling platform.
3. Huge undocumented bash scripts
Uptime expectations
Infrastructure migrations
Too many one off oddities
The Usual Suspects
Time… what’s that?
12. DO IT FOR THE
FUNNIES.
Something that started as a joke, ended with a signed quote
Got leadership to ask, “What’s the big deal?”
We got to involve the company in a fun charity project
Found a creative way to “poke the bear”
15. DO IT FOR THE
FAMILY.
“Special” nodes caused unexpected emergencies
“Off-hour” emergencies affect the staff & their families too
Your clients have families too
Stories of time stolen from families
19. DO IT FOR THE
COMPANY.
We all get excited to play with new toys, but when that new toy
gives you time to play with more new toys… priceless
Faster resolution time to some incidents
Faster roll out for new dedicated clients
A lot of human error goes bye-bye
20. DO IT FOR THE
SECURITY.
Yeah, it’s a real thing…
21. DO IT FOR THE
SECURITY.
Multi OS infrastructures pose complications
Patching in minutes instead of hours
Internal & External audit reports
We plan to see changes near real time
22. TIME…
There is a theme here…
… to explore possibilities
Unicorns DO exist!
… to explore the shadows
… to explore the real world
… for the things you SHOULD be doing
I do have to appologize and hope we have some new content in here...
Notice I'm here and not fighting a fire
ap for about 3ish years
and we provide an Enterprise level scheduling platform
150 nodes provides scheduling for pet groomers to state sponsored initiatives
Huge undocumented bash scripts
had to meet unrealistic uptime expectations
Unrealistic uptime expectations
Shortly after I was up to speed we move the entire infrastructure from Rackspace to OnTop, a local provider
We had one off oddities across the board
And with only a two person team, no time to realy improve the situation
Many nights passed out in my office chair
went through free trials, expired quotes, discounted quotes & more expired quotes
over eager newbie
Director out of town
Sock puppet gestures in the middle of meetings where I’d bring up Puppet
Maintenance window sock puppet idea
Scary 4am puppet making or that it turned out pretty good
Idea to litter his desk
Time for payback
Gathered supplies
Sent out an email
Calling the troops to help our efforts and create a puppet for charity
Decent turn out
people swung by throughout the day and made a puppet
placing them everywhere from his phoneto his peripherals and the arm rests of his chair.
they were everywhere.
I think we ended up with around 50ish puppets.
Director came in to a good laugh and probably a little creeped
We got to do something fun for House of Hope, local shelter for working single mothers
But the whole thing got leadership asking questions and opened the discussion
Why all the trouble
Started as a joke, ended with a signed quote
My biggest reasons for pushing through all that for just under a year…
Wife and I took a camping trip
Camping trip, wave of service, nuke sirens
Dedicated client
Long story short
Sitting in my truck a couple miles away where hotspot could keep a signal
Worked issue, default php limit that wasn’t changed
Came back to this… Not a happy wife
Not what you want unhappy wife
vacation in New York
Similar thing happened
workaholic
covered in Puppet, its what it does best
We’ve all heard stories, that one is mine
Story of the unhappy wife… client staff wasn’t able to see their appts for the day,
Which he made it very clear we were taking time away from his family by him needing to deal
So all in all… if clients rely, issues can trickle downstream to affect clients and their families
Happy to report the unhappy wife is still my wife but is happy
And we’re actually going camping this weekend with this guy
Which makes him VERY happy and I’m not concerned that something
So simple is going to set of my phone into a frenzy
So of course you get executive buy-in and they expect some form of return...
faster resolutions if not, completely side step some of the issues we previously had.
dedicated stacks, the faster we get them implemented, the faster we get paid.
time to delivery has been cut more than half since our deployment.
A company unable to evolve, improve by implementing new technologies and tools, in my opinion is operating with a shortening lifespan.
More time to play with new toys, learn, and implement improvements
Another thing that can shorten your lifespan as a company...
something worth protecting, odds are you're doing it wrong
Integrity of an OSS project, sensitive client data
I’m taking baby steps to learning a bit of windows administration, linux admin after all
We’re patching faster min vs hrs
Able to provide internal & external audit reports via PE Console and mCollective
Ver numbers of popular service attack vectors
We are paying attention
New ways to acknowledge certain types of changes via Hipchat bot fed by puppet
We found a unicorn
Constantly working and pulling more out of the Puppet magic hat
Touched the slight void that was security
Time to develop new toys
I’m able to take my vacation without panic attacks for simple issues.
