4 minute read
Data intelligence is the ability to transform massive data sets into intelligent data insights using semantics, machine learning, and artificial intelligence technologies.
Such processes typically improve the services and investments that corporations have done in their data.
Corporations have various kinds of data sets within their ecosystem.
Some of the data sets are structured, and some are unstructured. For example, Word documents, PDF documents, and PowerPoint spreadsheets are all classified as unstructured data.
Circa 80% of the world’s data right now is unstructured. In addition, there are structured databases like relational databases where data sets are organised in a more normalised format.
Data intelligence starts with organising your data, classifying it, and then extracting information from that data, or rather data points, that give new insights or meaning to your data.
This kind of information extraction uses various machine learning models or fuzzy matching using custom ontologies and taxonomies.
Why is data intelligence particularly important in the context of regulations and compliance policies?
Corporations are trying to solve various use cases in compliance and regulatory environments.
Regulated industries have heavy reporting needs before compliance bodies. Typically, four use cases are common in the regulatory environment:
- Facilitating e-discovery and reporting on what they knew when they knew it or how they arrived at a decision at a given time. This also relates to time-stamping data.
- Enabling regulatory compliance. For example, for GDPR, there may be a request to remove all personally identifiable information the organisation doesn’t consent to.
- Ensuring automation, for example, for automated the report creation
- Increasing visibility, for example, on product performance or post-market surveillance.
There are various ways to make use of data-driven intelligence applications.
They include trader surveillance platforms, digital publishing platforms, various semantically linked regulatory compliance applications, cognitive search analytics platforms, and clinical research data hubs.
For example, the EPA—the Environmental Protection Agency in the agrochemical space—has set a 2035 goal to stop et al. mammalian testing.
It becomes increasingly important to build these intelligent data platforms so that companies can comply with such regulations.
How do we help our customers achieve data intelligence?
Datavid, as a company, is focused on building data-intelligent platforms.
We have built many tools and services that make it easier for organisations to collate their data, extract information from their data, organise and store the data in proper indexes, and build custom visualisations tailored to specific needs.
This approach improves the quality of data so that users can search for and find information using their domain vocabularies and share it with downstream analytics or reporting applications.
Essentially, it makes the data more structured, which allows one to report on the data easily to regulatory bodies.
We have expertise in proprietary and open-source NLP software in text extraction and table extraction.
We have built connectors for common repositories like SharePoint and common BLOB storage like S3, enabling more downstream use cases like data science and data analytics on extracted data.
Our expertise in working with unstructured documents and multi-model databases gives us an edge and ability to transform unstructured data into a normalised structured format, with section identification, entity extraction enrichment and searchability.
This approach allows us to build a data-centric application quickly.
What are the practicalities of working on a data intelligence project with customers? What is a typical composition of a team working on a project? What are the timelines?
Because we have already created some reusable micro-services in this area, enabling teams to gather structured and unstructured data and put it into a multi-model database, we really expedite the timeline.
We have an orchestration engine and expertise in AWS serverless technologies. We also have a multi-model NoSQL database like MarkLogic, which allows data to be loaded as-is and then converted into a normalised format.
Integrating with the underlying data repos might take two or three months, but we already have pre-built micro-services in this area, which we can do within a few days.
The timeline gets reduced by at least 50-60%.
With a small agile team of one BA, one QA, and two developers, we can really reduce the timeline from a one—to two-year project to three to six months.
What are the regulations we currently help our customers comply with?
As mentioned previously, there are a few use cases for data intelligent platforms.
One, of course, is Discovery.
Once you’ve indexed and enriched the data, you can discover and interpret it more effectively.
That’s also a very important use case for compliance in R&D industries, where there is a lot of research and development. In the Life Sciences and Pharmaceutical Industries, we have built applications and reporting that help with EPA and MDR regulations.
For example, under MDR, which came into force in May 2021, medical device manufacturers must adhere to strict guidelines to ensure the safety of their products.
To do this, they need to perform post-market reporting in a more granular and automated way. Data intelligence platforms help expedite the business needs around reporting.
There are other examples around GDPR and FCA.
How do we help customers accelerate time to delivery and time to value?
For this purpose, we have reusable micro-services to collate and organise data.
We already have the Alpha version of our platform, Rover.
If you were to summarise three main takeaways from this interview, what would they be?
It is important to draw a conclusion on why Data Intelligence Platforms are so important, and they can help with, including:
- Building new e-commerce applications which open new revenue streams
- Ensuring regulatory compliance and reporting needs.
- Providing substantial benefits and value by improving your workforce’s efficiency and productivity.
I would encourage you to talk to a Datavid Consultant, and we can help!
You can also watch a recording of our webinar, which served as the basis for this interview, at this link.
Frequently asked questions
