I’ll just say it; I’m proud of our release cycle for OLS.Switch.

It has been my experience (YMMV) both first hand running an authorization host/switch (issuing and acquiring) and as an IT Security Auditor and QSA - that either Core Banking applications or Payment Switches fall into one of the following when it relates to upgrades, changes, or security updates:

• "The Vendor set it up, we don’t touch it"
• "We don’t patch it because we are afraid"
• "We cringe everything we need to install a new release of the software"
• "Last time we did an upgrade, we had x amount of downtime"
• "It all goes smooth like clockwork" 

During Vulnerability Assessments and Penetration Testing on the Internal Networks that I performed– My observations from an operating system, database and application perspective - these systems are typically not keep current or run on a platform that the organization is not very familiar which and relies on outside support. The application was not cohesive to the rest of their operating environment: systems, technologies, and procedures.

Installing new releases of our software (or rather our clients installing releases of new software) is something that does not make me cringe. (and I used to not sleep very well in the past)  At least one of our clients seems to agree as well. (See Andy’s "A very simple platform to support")

We just rolled out a new release that was quite large (see Flexible Spending Accounts (New Initiatives, Part 3) and had changes that impacted pretty much every transaction path due to partial authorization and credit reversal support, and required heavy regression testing. Our agile based SDLC is a big help with this, we have very iterative development processes and frequent testing which also means less large bulking updates that break everything.

Another success factor is our simplicity of upgrading our program code and binaries. It is really as simple as:

• Stop the OLS.Switch Service or Daemon
• create a backup copy of the directory or file path where OLS.Switch is installed
• Start the OLS.Switch Service or Daemon
• Perform test transactions and monitor.
• The Back-Out plan is to stop the service and revert back to the backup copy.

Further, System Implementation Design can have a big impact on Up-time;  we run multiple independent application servers behind load balancers, that allows us to gracefully stop an application - which stops accepting new transactions when finishing to process those in its queue, and the load balancer stops routing transactions to this application server. Allowing an upgrade to be made while other application servers are still processing transactions. Uptime, not system uptime, but uptime processing transactions doesn’t have to suffer for "scheduled maintenance", or security related patches and reboots.

I think we have a low-risk upgrade/update path that our clients are very comfortable with - So in 7 months into the year - we have had a dozen releases to add additional functionality, address endpoint changes, and implement new transaction types.

I was thinking about the tools and systems and general knowledge that I use on a daily basis, and thought is would be a good exercise to document them here:

Computer Systems:

• Quad Core Q6600 Vista 64 Workstation 8GB of RAM
• Apple Macbook with 4GB RAM
• VMWare Workstation and VMWare Fusion (running various linux distros and Win XP images)
• Large Monitors 24″ +

Software:

• A good text editor - UltraEdit, TextMate or vim
• A good hex editor - UltraEdit, hexdump, hexedit or vim -b
• ssh client - PuTTY or OS native ssh clients
• svn client
• NetCat - to pipe binary message dumps to simulators, and to create listening servers to accept message dumps.
• Client and Server Simulators — write your own!
• Calculator — the DEC to HEX and HEX to DEC functions are great for header lengths - if you are a real geek you have a HP-16c
• Instant Messenging clients and Skype to communicate with your team.

Knowledge:

• Ability to read — seriously, read those specs!
• ISO8583
• TCP/IP Socket programming - both Client and Server
• Database programming experience e.g. SQL - and other O/R Mapping tools - Understanding the I/O requirements between write intensive OLTP and read intensive Data Warehouse data stores.
• Payment Processing 101 knowledge
• Data Encoding techniques and character sets and numbering systems - ACSII, EBCDIC, BCD, Binary etc.
• Basic understanding of Encryption, including symmetric, asymmetric, PIN encryption , PIN translation, and DUKPT
• Basic understanding of IT Security
• PCI and PABP/PA-DSS requirements - review the audit procedures !
• How to use Google
• Ability to read — Note: This is intentionally listed twice