All around the globe, organisations store data in whatever form possible: images, genome sequences, clinical data or even actual samples, e.g., frozen biopsies from a tumor. These data could be valuable for researchers. However, the process of obtaining these data can be a big burden.
Most of the time the request process is a long process and consists of a series of manual steps such as:
- searching for the organisation which has the data you want,
- sending an application form,
- waiting for getting approvals
- and finally waiting for the data to be delivered
These steps are not only time consuming due to waiting times. It is also hard to get a good overview on where you can retrieve each type of data and what the status of your application is. Especially when multiple organisations or biobanks are involved for a particular research question.
In 2015, The Hyve developed a request portal for PALGA, the nationwide network and registry of histo- and cytopathology in the Netherlands. The goal of that development project was to migrate their request process from manual to a semi-automated process. However, it was only specifically tailored to PALGA’s needs. Based on this portal, the Dutch consortium that is working towards a national biobanking infrastructure (BBMRI-NL) asked The Hyve to implement a generic request portal which would be a central request portal for:
- requesting samples from a biobank
- requesting data from a registry
- requesting images from neuroimaging centre
We named this open source project: Podium.
What did we do?
The project started with creating a set of potential solution directions, based on the initial requirements described by BBMRI-NL, and taking into account our experience with the PALGA request portal. How should the solution function? Should it be centralized or distributed among different organizations? What hurdles do we expect based on our experience?
After this was done, we investigated these possible solutions. How would they function? How good is the scalability? The customizability?
We determined which solution would be the best fit by picking a scenario, such as “Every instance deployed for each institute will have its own user interface (UI) but all backend (API) communication will be done with a single master (scalable) server”, and running different use cases through it end-to-end. We made a comparison of different aspects, in particular performance and scalability, and decided to go with the best scoring scenario.
When we determined what would be the optimal solution direction, we moved onto which technologies and frameworks we should use. We looked if the technology was (of course) open source and if it could support the scenario we wanted. After that we started the actual development in an agile process, frequently demonstrating the state of the product to the product owner and key users for immediate feedback, and then improved or altered the components based on the feedback.
What we came up with
The main purpose of Podium is to become a generic request portal, one which is not tailor-made for a specific organisation, but supports request processes for multiple organisations. Therefore, the ability to customize the request form and flow of requests was important to keep in mind for the future, while addressing the first needs for BBMRI-NL.
By first creating a template where each part has its own individual programmatic component, the blueprint for the road to being fully customizable was laid down.
The most exciting part programming-wise was creating an application based upon microservices. This is an upcoming variant of a service-oriented architecture. Each component serves a clear purpose with a clearly defined protocol. This makes it easy for these services to communicate with each other, distribute the workload and let the microservice become part of something bigger than themselves.
The principles of microservices are explained in this graphic: (taken from: https://martinfowler.com/articles/microservices.html)
By connecting different microservices, each serving its own purpose such as handling the request, Podium will be well-prepared to become a scalable application beyond Dutch biobanks and registries, for example in a Europe-wide setting without being bothered by performance issues.
Researchers can choose whether they want data, images or material and from which organisation(s). Eventually, when they submit their request, keep track of the status. When an organisation wants to make their data available for request, not only the organisation itself, but also the specific data needs to be findable. This will now become a matter of having your options inside of a request portal, on the stage, on the Podium.
All these processes had to be done manually beforehand. For each organisation or biobank a researcher needed to find where they can fill in a request, what data they actually can request and keep track of it manually, while this process could take up months… Making errors during such a long and often cumbersome process is naturally and not necessarily avoidable. Did the organisation even receive your request? With Podium you can easily keep track of these requests and what their status is, so you can focus on your research.
In summary, Podium is an inclusive, extensive, scalable and customizable request management portal.