By Joe Timmer, Director of Engineering at AdvicePay
Jumping into the middle of an ongoing software project is a lot like taking children on an airplane. It’s exciting, but nobody involved really knows how it will turn out, and the people nearby watch with trepidation.
I heard from Alan on August 9th, 2017. Alan’s pitch was basically – “Let’s create something meaningful in a sustainable way, where we don’t burn out everyone involved.” I felt good about that vision, and we kicked things off a month later.
Nobody on the team, including myself, knew how to organize the initial team which included myself and a contracted development team of one person, at that time. So, this has been an amazing adventure with lots of crucial decisions and turning points.
Where to Begin
There is a necessary bridge that must form between the vision holder for a piece of software and the team that builds the software. This had not happened for the AdvicePay project. It had already seen a launch where nothing was wired up. It looked okay, but the actual functionality had not been built.
Basically, once I got onboard, it was do or die. The codebase itself was a single directory with a slew of files. This is kind of like trying to build a house when all of the tools and materials – nails, wood, hammers – are in a single pile that you have to sift through to find what you need for the task you are currently working on.
It’s a small miracle that we are now processing over $1mil/month. This is the story of how AdvicePay finally got off the ground and why the team that got it there is the essential ingredient.
There was direction that came from Alan and Mary on where they wanted to see the project head, and there was the reality that I saw in the code. For example, emails were just plain text. They looked exactly the same as an email from the Prince of Nigeria whose daughter needs some extra cash to go to college. This needed some love if we were going to build enough consumer trust to manage folks’ money.
The existing structure of data also was scary. We weren’t tracking invoices created on a subscription. This meant that for many places where a charge occurred, there was no correlating invoice. This didn’t seem like a project that could have real users – but it did! This was the most basic shit needed to give people a payment processing tool. The list goes on. Subscriptions could not be canceled. Failed payments could not be retried. I never really had time to be overwhelmed or question if we should continue onwards because there was so much to do.
Formulating Our Team
On top of all this, our team structure was extremely confusing. I was hired as a junior developer to help with front-end development. According to Alan, the backend was complex due to our use of Stripe. We not only had to manage the state of our data, but also that of Stripe. However, I started writing backend code close to day one, as the need was so high, and cranking on the front-end was not going to get us where we needed to go.
If it’s not outrageously clear, like most early stage projects, there were a lot of things that needed to go right for us to get off the ground. We grinded on the product for a couple months, and we were ready to launch by the beginning of 2018. In getting to this step, the real work was just beginning –to give people a secure platform that could be relied upon — and we desperately needed some more people.
Kevin was fresh out of a programming boot camp, and though we could have used a more senior team member, I could see that Kevin would stay focused on something until it was completed, no matter the effort required. I knew there was still a lot to get done, and I wouldn’t have the adequate time to train a junior developer. We could either wait to find someone with more experience, or we could hire a Kevin. I am so grateful that this worked out the way it did. Kevin’s determination and good attitude when he didn’t have the necessary resources to do his job well are part of what allowed us to go from launch to reliable. This is more difficult than getting launched.
We were able to get our minimal viable product to market, but what about safeguards to make sure we don’t move a decimal point in the wrong direction or charge someone multiple times? How do we ensure that if we make a mistake, that we don’t make that same mistake repeatedly? How do we get ahead of our customers so that if they experience an unexpected error, we know about it in real time? On top of improving the functionality and adding new features, we also needed to improve the way that we built features to ensure high quality releases and awareness of any issues that we did not think through.
Adding Unit Testing
In programming, there is something called unit testing, which allows you to write a test in code that will check for an error. For example, what happens if a user inputs $0 into the amount field when requesting an invoice? First, who would do such a crazy thing? I wish I didn’t have to think about such cases that do not make sense, but as a programmer, we need to account for everything that a user could possibly do with our software and protect them against themselves. This is not an easy task.
Tests are a big win in dealing with these edge cases. They can be difficult to write, but Kevin took on this task with amazing focus. Not only were we reliably processing money, but we were able to write tests for the existing code, and add tests when we encountered issues in production. This enabled us to respond with agility, while also ensuring we were never making the same mistake twice. We could not have made the transition from launch to reliable without Kevin.
Improving the Core Product
I had spent a couple years prior to working on AdvicePay doing contract work. At almost each project that I worked on, there would inevitably be the question, “Hey, do you know any other programmers?” Invariably, I would bring Isaac onto the project, and after a couple weeks, he would impress the project manager with his ability to get more done than their full-time programmers while only working part-time.
As a programmer, I’ve always been impressed by Isaac’s voracious appetite to write large amount of code, but his thoughtfulness and care is what has constantly won me over in regard to Isaac’s working style. He joined the project a couple months after I did, and we consistently nudged him to move to full-time. He is a cautious individual and wanted to see if the project was going to still exist in six months. Well, we still exist, and Isaac joined the team full-time in the summer of 2018.
Becoming More Proactive
At this point, things felt 10x better than where we started with Nigerian Prince emails and missing most basic functionality one would expect from their payment processor, but there were still some gremlins that could bite us at any moment.
We were building all of this great stuff on top of a database called Mongo. Mongo has a purpose, but if you talk to most programmers, they will rant about why Mongo is a bad choice. Mongo gives no guarantee about the structure of your data so as the project changes, the only way to really know how the data exists is to read all of it (hundreds of thousands of lines of data).
I won’t get too technical, but to build software, you have to know how the data exists so you can create reliable software. Having all of our core business logic on top of a schema-less (no formal structure) database was a disaster waiting to happen. The longer we waited to remove Mongo, the harder it would be to make the transition.
Goodbye Mongo, Hello SQL
We had been working on a major piece of our technology, digital contracts, at the time that this database issue was garnering more and more of my mind space. We finally decided to migrate to a SQL database in July. I had a baby coming in August (as did as 50% of our AdvicePay team!), but felt that it was crucial to make this shift as soon as we could.
In January, we had hired Isaac PT to build out new features. He had been consistently churning out high quality work, but swapping databases while making sure our existing data doesn’t get bonked is no easy feat. This process requires a technical write-up on its own, but to give the gist, we created a single data migration script to create a new database and move the existing data from mongo into our new SQL database. This was ready a couple weeks after my son was born. Isaac and I shared a video call while this went live, and there were zero headaches. I don’t know how to illustrate how important this is. This is when having talented team members is the difference between success and epic failure.
Migrating from Mongo to SQL feels like a lifetime ago, and we’ve been able to keep growing and improving our own internal process. The thing about starting this project with Kevin and Isaac is that they are unique in their ability to own certain features with minimal oversight and they set the bar for any additional team members. We will continue to protect and grow our team with a fierceness toward protecting our ability to move money reliably from point A to B.
We’ve come a long way since then.
AdvicePay’s development since these early moments has been constant and aggressive. Our monthly releases have pushed out some pretty nuanced features for our users, as well some complex integrations, like eSignature and others.
Want to follow our Release Notes? Subscribe to our blog to begin receiving these updates and more from AdvicePay.