Industry
Standards Organizations
Challenge
Standards Australia aimed to modernize its publishing workflow by adopting XML, migrating from ISOSTS to the more advanced NISOSTS data model, and transitioning its systems to the Azure Cloud—requiring major changes across content management and upstream tools. However, despite extensive development, the project faced performance bottlenecks, inefficient XML chunking, legacy code, lack of unit tests, and architectural complications that delayed production readiness.
Solution and Results
Datavid quickly identified the root causes of performance issues and fully rewrote the core application, optimizing data models, search architecture, and deployment pipelines. Through a series of targeted interventions and PoCs, the team dramatically improved system performance, enabled successful UAT in the cloud, and supported a smooth go-live—paving the way for scalable, standards-aligned digital publishing.
Technology used
Progress MarkLogic, Angular, Azure Cloud, Microsoft.NET
About Standards Australia
Standards Australia is the national body responsible for developing and maintaining Australian Standards, ensuring quality, safety, and efficiency across industries. Founded in 1922 with the name of Australian Commonwealth Engineering Standards Association, Standards Australia is the country’s leading independent, non-governmental, not-for-profit standards organisation.Setting the Scene
For standards organizations, delivering high-quality, reliable content across multiple formats and channels is essential to supporting innovation and industry compliance. But legacy data models and outdated infrastructure often slow down development, limit product functionality, and increase operational overhead.
Standards Australia recognized the need to modernize its content systems to support faster publishing cycles, enable richer product features, and align with international data standards. To that end, the organization launched a new XML-driven search platform, SIM NISO, built on the NISOSTS data model, an internationally recognized standard already adopted by ISO and IEC.
Initially developed over 18 months by a third-party vendor, the SIM NISO platform struggled with significant performance and stability issues that prevented a successful go-live. Faced with mounting delays and technical setbacks, Standards Australia turned to Datavid, EMEA Progress MarkLogic Partner of the Year, 2023, 2024, to get the platform back on track, with a sharp focus on performance, maintainability, and readiness for production.
The Challenges
Standards Australia wanted to implement a program to adopt XML. The use of XML offers many advantages to standards organizations, such as easier multi-format and multi-channel publishing, better internal and external linking, higher content quality, and simpler integrity checks.
Furthermore, a well-structured XML publication workflow can also reduce publication time and enable the introduction of new products and functionalities (e.g. automatic display of differences between two versions of a standard).
SA also wanted to migrate from ISOSTS to NISOSTS data model, which is already used by other international organizations for standards development such as ISO and IEC.
NISOSTS data model:
- Has much greater metadata capabilities;
- Enables more visualisation-focused modelling of XML reducing the burden of creating style sheets for rendering in multiple environments;
- Offers a new recursive model for adoptions;
- Accepts both XHTML and CALS tables, and both MathML2 and MathML3 (ISOSTS only supports XHTML and MathML2).
In parallel, Standards Australia decided to initiate a cloud migration project aimed at seamlessly transferring all on-premises systems to the Azure Cloud.
As a result, the project is set to transition to a cloud environment, encompassing the execution of User Acceptance Testing (UAT) on the Cloud UAT environment.
To achieve this, the project team was tasked with constructing the entire spectrum of environments, ranging from the foundational lower environments (Dev and Test) to the more advanced higher environments (UAT and Prod) within the Cloud infrastructure.
Additionally, they needed to establish and test a robust CI/CD pipeline, while effectively relocating MarkLogic Database Servers and Application servers to the Cloud.
To take this project forward, some changes were made:
1. An upgrade of their standard content management system from the current ISOSTS data model to the NISOSTS data model to allow ingestion services to input ISO and IEC data into the NISOSTS format through harmonised APIs.
2. Some changes in all their upstream systems that contribute to the development of the standards, so that these systems and tools upgrade to the NISOSTS format.
After extensive development effort, the new platform still suffered from application performance bottlenecks, and the cloud migration introduced additional hurdles.
Despite significant investment, the platform was not ready for production and could not be released as planned.
The second phase of the project moves instead to UAT and integration testing with upstream and downstream systems. The development methodology is agile, with 2-week sprints monitored via JIRA and Confluence, adopting CI/CD deployments.
The following challenges were identified:
- Performance problems and poor improvements.
Chunking of the searchable xml file was inefficient and led to little improvement in performance.
- "Dead code"
Dead code refers to sections of code within a program that are written but never executed during runtime. These code segments do not contribute to the program's functionality or produce any meaningful output. Much of the remaining code still refers to the non-chunked approach.
- Absence of unit tests.
Changing anything in the code would have to undergo a manual testing process, whereas, with unit tests covering to some extent the testing of the code base, this refactoring would be automated. In general, the lack of unit tests increases the cost of system maintainability.
- No module database.
This provides an unnecessary complication of the deployment pipeline, which may have an impact on the performance of the current system.
- The repository is shared for multiple projects.
The Solution
Datavid followed these steps to find a potential solution:
- Get access to real data from UAT environment
- Start implementing the proposed solution before UAT
- Implement ml-gradle fully with code in the modules database
- Use UATs to actually test the changed code for the services
- Fix any bugs coming out of the rewrite during the UATs
Within just one week, Datavid diagnosed the root causes of the performance issues, including a poorly designed physical data model, inefficient fragmentation policies, and suboptimal code structure.
The team then fully rewrote the core application in just six weeks, using one full-time consultant and a 0.5 FTE technical lead. During this period, Datavid’s QA team conducted performance benchmarking to validate improvements and ensure readiness for production deployment.
The final solution proposed by Datavid involved:
- Eliminating chunking of searchable documents.
- Creating indexes for filters and query facets.
- Remodeling the template-driven extraction (TDE) based on a single searchable XML document.
- Remove Optic API from the code and replace it with cts:search, starting with the least performing services.
- After removing Optic API, also remove the TDE.
These performance improvements were implemented in three interventions, each preceded by a Proof of Concept (PoC).
The Outcomes
Datavid improved the performance of the system, using three strategies for a solution. The third engagement looked specifically at the technical architecture for search and implemented a new solution.
The cumulative effect of these performance improvement approaches enhanced the request performance from about 10 seconds to about 300 milliseconds per request.
In the end, Datavid contributed to:
- Improve and optimise system performance in Cloud environments.
- Optimise the quality of code.
- Make recommendations on improvements/refinements needed to the MarkLogic technology stack and topology.
- Build, configure and optimise Cloud environments in time for UAT
- Automate data migration where possible.
- Improve data synchronisation and data quality between the NISO SIM production and test environments.
These interventions led to a dramatic improvement in search performance.
- Previously, 1 user performing a search experienced 40 seconds of delay
- After the rewrite, 20 users can now search in under 1 second
Thanks to Datavid’s fast, expert delivery, the customer was finally able to go live within three months of the team joining the project.
The key results are:
- The review of:
- the NISO UAT environment and technology/topology stack to identify areas for performance improvement at the environment creation and technology/topology stack level.
- the quality of the source code/code base to identify gaps and performance losses.
- Assisting the SA Tech Lead in building and configuring Cloud environments and providing guidance on best practices to achieve efficient, reliable and high performance Upper (UAT and Prod) environments.
- Identifying the right Load Balancer for Upper (UAT and Prod) Cloud environments.
- The definition and adoption of the data migration strategy.
Want to learn how Datavid can help modernize your content platform?
This transformation not only enabled a successful go-live,but also set
the foundation for scalable, standards-aligned digital publishing.
Curious what’s possible in your environment?
