By Jacques Marais – Team Lead and Architect of the Helium Platform Development Team at Mezzanine | First published by Offer Zen on Wednesday, 25 November 2020.

Software developers prefer to not solve the same problems over and over. This mindset coupled with a business requirement of being agile and efficient is part of what led my team at Mezzanine to develop a rapid application development platform called Helium. Building our own platform came with a lot of challenges, so I’d like to share how we approached it and what we learned.

When I joined Mezzanine, most of the problems we were solving were focused in the domain of mobile health. The applications we were developing were mostly crud-based web and mobile applications with some complex back-end logic for notifications and other asynchronous processing.

At the time, there were many application development frameworks available that we could potentially use to develop apps. However, none of them provided us with the simplicity and flexibility to work with that we required, nor did any of the available frameworks have the domain specific elements that we preferred.

Developing our own platform seemed like the logical answer to this problem: It would allow us all the flexibility we required and would allow us to adapt the platform as our requirements, and the requirements of clients, changed over time.

As development of the platform started, it became clear that there were three main challenges that would need to be solved in order to be successful. We had to:

Here’s how we approached solving these challenges, and a few of the main lessons we learned along the way.

Challenge 1: Determining which components were similar

The first challenge we faced was determining what functionality was common to many of our applications, and what core features the platform should provide. These features needed to be implemented in a more general way and eventually exposed to developers somehow.

Since we had already developed various applications, we had an idea of the features that were common to our applications and therefore what needed to be implemented as core features in the platform. We also needed to anticipate what might be needed in the near future.

Some of the key requirements that we identified were as follows:

Once the requirements were identified, the core functionality was implemented using Java EE and encapsulated as separate components that would be exposed to application developers.

The result was a feature-rich back-end that could leverage the functionality and architecture provided by Java EE, with the aim being to provide developers with a complete and simple to use platform. With the core back-end functionality implemented – but not yet exposed to developers in an easy to use package – there was still work to do.

Generalisation does not mean less complexity

Because of the fact that the reusable components in Helium need to be generalised for potentially many different (but similar) use cases, it requires a considerably more complex implementation to ensure that all edge cases are covered. This means that adding new features to the platform is complex and potentially time consuming.

The so-called black box approach of the platform also has a disadvantage. In cases where the system does not behave as expected or an issue is encountered developers might not be empowered to solve problems themselves. This potentially results in high support costs.

Challenge 2: Exposing these components to developers

With many of the back-end features implemented and abstracted away, developers needed to be able to use these implementations in their applications.

At the core of this challenge, the goal was to have a platform that could provide a way to rapidly develop applications. As such, we needed a way to expose all of the back-end features to developers in a way that would help developers build apps faster.

We specifically wanted to avoid a situation where developers needed to know the intricacies of the Java EE ecosystem in order to be productive. We wanted the platform to take care of those aspects so that developers could focus on the business logic of their applications. In addition – and because our applications were domain specific – we decided that a domain specific language (DSL) would be appropriate. The result was the Helium DSL.

The DSL is a programming language with related tools developed in-house specifically for developing the types of applications that Mezzanine develops and expects to develop in the future. It is implemented as a model view presenter (MVP) framework using XML for specifying views and a custom C-like language for specifying data model objects and presenter logic. This is helpful because the DSL allows developers all the functionality previously mentioned, without a need to know the intricacies of the underlying implementations.

With the DSL in place, the Helium server could be used as an execution/hosting environment supporting a multi-tenant architecture. In other words, it could be used as a single server instance that could house many different applications.

The advantages were easy to see: Any improvements to the platform implementation were immediately available to apps running on the platform. Solving a problem for one application would allow all applications running on the server to benefit from the solution. The result was a platform that satisfied many of our initial requirements.

Here’s what we achieved:

Database integration

The DSL provides syntax for specifying model objects along with internal ORM functionality. It’s as easy as specifying model objects and then using presenter logic and calls to built-in methods to save or query from the database.

To provide developers with even more control, native SQL can also be executed using built-in functions. The code snippet below shows a simple example of persisting and querying data using these methods.

User management

Objects specified in an application’s data model can be marked as a user type of specific role using a provided annotation. The DSL provides built-in functions that can then be used to invite users for that role. A single user can be invited as different types of roles and easily use built-in UI components to switch between roles.

Model objects can further be annotated to only allow access to that object for certain roles using custom criteria.

Cloud-based

Since the Helium back-end is implemented in Java EE, this requirement was easily satisfied as part of the tech stack and execution environment.

Hosting concerns

The Helium platform was implemented using a multi-tenant architecture that would allow many apps to be deployed and run on the same server. Developers therefore do not have to concern themselves with the underlying architecture and implementation details of the hosting and execution environment.

Simple UIs

The DSL facilitates UI development by providing a variety of components out of the box. These include simple components for data capturing and complex components such as data tables, maps, an image gallery, and more.

The MVP architecture along with these UI components, a fixed layout scheme, and fixed uniform styling means that developers have flexibility in what UI components they want to display and how these should interact with application logic without having to spend hours on other aspects of the look and feel.

Integration with external services

Many external services can be called from within DSL applications by simply making calls to provide built-in functions. Initially these included functions to send SMS and e-mail messages but later enhancements also included calls to Mpesa and general outbound APIs.

Developing a programming language is not easy

A lesson that we learnt was that developing a programming language from scratch is not easy. In part, this was expected. What was slightly more of a surprise was the difficulty in expanding and maintaining a programming language. The result was a somewhat slower turnaround time in developing new language features for the platform than expected.

We also realised that there is a constant balance that needs to be maintained to avoid expanding the DSL with too many general programming language (GPL) features that would essentially change its nature from a DSL to that of a GPL.

Allowing for very quick development will always be a big advantage of the Helium platform and the Helium DSL; but, with any framework or platform, there is some compromise of flexibility. This means that it’s possible that the platform approach adds complexity for certain use cases instead of reducing it.

With regards to hosting apps and utilising a multi-tenant architecture, additional challenges were faced. Since apps are sharing resources and any misbehaving apps have the potential to bring down the hosting and execution environment, a considerable dedicated effort needed to be made to ring-fence apps in such a way that this did not occur.

Challenge 3: Providing the set of tools for stakeholders to work with

With the Helium DSL and back-end in place, the final piece of the initial puzzle was providing the tools for both developers and non-developers to deploy, manage, monitor and configure running instances of their applications.

A key requirement here was a core web application that could provide authenticated users the ability to, amongst other things, create and configure apps and to upgrade apps with previously released source code. Developers also needed a simple way to build and release source code to a Helium server.

These requirements were met by building two stand-alone tools that would integrate with the web services provided by the Helium back-end:

Collectively, the Helium DSL, execution/hosting environment, all backing services and tools are now referred to as the Helium platform. The tools essentially provide many different users the ability to interact with the platform in different ways.

They also allow developers a short turnaround time when developing and deploying applications that further satisfies the original requirement of being agile and able to react quickly.

Tools like this are key for accessibility and development

In general we realised the importance of these types of tools and how, despite not being part of the core back-end and DSL, they provide essential features required to deploy and develop applications. The tools also demonstrated the extensibility of the whole platform, made possible through integration resources in the Helium back-end.

As with any complex software project, the first iteration is rarely a success that satisfies initial expectations. This was the case for us. There were two predecessors to Helium that were focused on many of the same ideas, but eventually did not provide us with all that we needed even though each iteration represented a step in the right direction. This highlighted the complexity of what we were attempting to achieve.

Given everything that we’ve learned in developing and using the Helium platform up to this point, the future for Helium is very exciting. Although we’ve had to re-evaluate some of the details, the general idea of providing reusable components is still very much relevant and something that the future strategy will rely heavily on.

With the complexity of a monolithic multi-tenant platform such as Helium and the potential limitations of a DSL, we are looking forward to an approach that provides a framework for development instead of a single platform.

Jacques Marais is the Team Lead and Architect of the Helium Platform Development Team at Mezzanine. Jacques is passionate about software development and using technology to solve real world problems. In his spare time he enjoys photography, hiking and growing his own vegetables.