Upgrading: Building for the Future

At the heart of RapidSpike is a development team who are passionate about pushing their abilities, learning new technologies and ultimately driving our software and product forward. We’ve researched and developed many cutting-edge features over the last 6 years but we’re shifting our focus this year. We’re taking a step back from new feature development in favour of upgrading, exploring and applying new technologies to existing functionality. We’ve learnt many things over the years so this financial year’s Technology strategy is to take that and push everything forward. Hindsight is a wonderful thing.

Recent Changes We’ve Made

So far we have rebuilt our:

  • Entire front-end sales and marketing website
  • In-app report generating and scheduling engine
  • Google Lighthouse monitoring system
  • Added the Chrome User Experience Report (CrUX) data

The website has been rebuilt in a way that makes managing, maintaining and publishing to it many times easier than it was previously. This has enabled us to react faster to new requirements and messaging updates and gave us the opportunity to design with performance and SEO considered first.

Our in-app reporting engine has been rewritten from scratch. Previously every report was generated separately, some with scheduling capabilities, some without. We’ve created a highly scalable, reporting module which supports scheduling any report out-the-box. To the user, scheduling is one of the only upgrades this has brought, but to us, it will enable us to develop reports much faster and to a better standard.

Our Google Lighthouse monitors used to run in Docker containers, store its headline data in MySQL and dump full audit outputs into AWS’s S3. We then ported this to a very basic Lambda function, which brought with it many gains, but we’ve now taken that one step further and implemented a completely serverless Lambda and DynamoDB stack. This has enabled us to stay on top of new releases, deploy to multiple locations and provide a stripped-back version to measure your website’s WebVital metrics. We’re not stopping there though – see New Technologies.

Finally, website CrUX reports have been added to round off the WebVital metrics we collect for websites. Google considers Lighthouse their “lab data” (controlled conditions) – we would take that one step further and consider our WebVitals data part of that lab data too. Google then provides their CrUX data as the “field data” (variables, from the wild) – this is sample data collected from Chrome users who have opted to share their browser’s performance data. We now ingest CrUX data from Google for our client’s websites and combine that in single dashboards to complement the Lighthouse and WebVital metrics.

New Technologies

Having already moved some services to Lambda functions and stacks, we’re now looking to see if other areas of our system can follow suit and be migrated to new cloud-native technologies. For example, we’re investigating running our Selenium grids serverless, our Webpage Tests using Puppeteer in Lambda functions and looking into upgrading to new, cutting edge data storage options.

Synthetic User Journey Improvements

Selenium v4 is right around the corner and we intend on upgrading as soon as viably possible. This will bring many new features both for us and for our clients. Most notably better grid management, new relative selector options, better window and tab controls, and some performance improvement. Video support out-the-box has also been promised which should mean we don’t have to rely on additional software layers and larger, more expensive servers, so we can offer this capability to a wider set of accounts.

We also have a whole swathe of ideas for upgrading our scripting layer too. This will be to standardise and streamline the writing process and make debugging easier. Our goal is to build the world’s best scripting engine with the lowest bar of experience entry. We will build on our existing functionality that enables people to write industry-standard automation scripts without needing any more expertise beyond understanding website structure and hierarchy.

Enterprise-Focused Features

As we’ve grown we have inevitably picked up more and more large, enterprise-level customers. This has brought with it requests for things we’d never considered before such as direct data ingestion through fire hoses, SSO integrations for user management, advanced reporting requirements and even ad-hoc Lighthouse testing for integrating with CI/CD pipelines! Despite being quite bespoke requirements, we will always endeavor to make as many of our features available to all of our customers as possible.

The first of these planned upgrades will be added SSO support in order to enable federated user access to customers who use IDPs such as OneLogin, Okta, Google and Azure, whilst also providing SAML and OpenID Connect capabilities.

Conclusion

This year’s focus is very much around improving and upgrading what we already have. Whilst we are always open to new ideas and features, we’ve reached a feature plateau whereby most requests we hear are more around additional configuration options, extended data reporting and integrations with other services that our clients rely on. We hope that by building for the future and our openness to new technologies will help build confidence in our commitment to deliver the very best, cutting-edge website monitoring platform we can.