Data quality and health database preparation

Because the quality of any analysis is capped by the quality of the database behind it, and a flawless analysis of an unaudited database can produce a result that is convincing and wrong. This applies to any study that starts from data already collected, such as registries, encounters, laboratory or follow-up data, before conclusions are drawn. Undefined variables, values that are possible separately but impossible together, outliers handled carelessly or uncharacterised missingness propagate silently to the final result, where they can no longer be told apart from sound data. Auditing and preparing the database is not a preliminary formality: it is the phase that decides whether the analysis means anything. The work also leaves the database documented and traceable, so that any later result can be traced back to the source data. It is the difference between analysing on firm ground and analysing on untested assumptions.

As well as checking that each variable is within its range, we detect conditional implausibility: a value that is admissible in its column but incompatible or clinically implausible when cross-checked against another. This applies to health databases, where the most persistent errors are not the ones that show up in a single variable but the ones that only emerge when fields are related to each other. For example, a BMI that is possible in an adult but impossible at a given age, or dates that are valid separately but inconsistent with each other. The work covers completeness and internal consistency, coherence between related fields, duplicates, and documentation of the data dictionary, with each variable's definition, coding and rules, together with traceability from the raw database to the analytical database. It is there, where variables cross, that the inconsistencies that survive a superficial review tend to appear. Checking ranges is the minimum; conditional implausibility is what sets a real audit apart.

Not by default: an extreme value may be a recording error or the most relevant finding in the study, and telling them apart means looking at the internal logic of the dataset, not applying a generic threshold. This applies whenever outliers appear, where the automatic decision, removing whatever falls outside a standard cut-off, can erase errors and legitimate information alike. Outliers are identified according to the actual distribution of the dataset, and the strategy is tailored to the case: retention, review, justified exclusion, winsorisation or advanced imputation when the value is judged unreliable and the analysis justifies it. The key distinction is between the outlier that is a recording error and the one that is a legitimate extreme value, and the decision is documented either way. Removing an outlier without understanding where it comes from turns an analytical decision into cosmetic cleaning. What you receive is a justified, traceable treatment, not a template-driven trim.

They are not simply "filled in": first we study why what is missing is missing, and the handling strategy comes out of that study, not the other way round. This applies to almost any real database, where dropout, missed visits or incomplete fields are the norm, not the exception. We assess how plausible different missingness mechanisms are (MCAR, MAR, MNAR) and whether the missingness itself may be informative, because data missing not at random can bias the analysis if they are handled with techniques that do not respect that mechanism. Depending on the nature of the data and the planned analysis, the strategy may be complete-case analysis, missingness indicators, sensitivity analysis or multiple imputation where justified. Imputation is a documented technique with explicit assumptions, not an invention of data the client did not collect, and it does not replace the clinical decision on which data can be recovered. Studying the pattern of missingness is what stops a standard technique from masking a bias instead of correcting it.

Yes, when the sources can be linked on defensible criteria: harmonisation and integration into a single analytical database are a core part of the data quality and health database preparation service. This applies when the data sit in systems that rarely speak the same language: clinical registries, laboratory systems, follow-up data and administrative databases with different codings, formats or vocabularies. The work merges those databases into a common structure, links records across sources using documented deterministic or probabilistic rules, and standardises variables to a consistent framework, with the mappings recorded. The result is a coherent analytical database with its data dictionary and mappings, which remains an asset for your team, not an opaque merge that cannot be audited afterwards. What determines the scope is the quality of the identifiers that allow the sources to be linked. When the link between records is ambiguous, that limitation is documented and handled explicitly.

No: the data quality and health database preparation service leaves the database audited, prepared and documented; the statistical analysis that answers the research question is the next phase and is covered by its own service. The boundary is worth clarifying because preparing and analysing are two different jobs, even though they are linked. Here the analysis is diagnostic before it is inferential: the quality of the database is quantified, missingness and outlier patterns are characterised, and the preparation pipeline is left under version control and reproducible on future updates. From that prepared database, biostatistical analysis, a predictive model, clinical registry analysis, RWE or any other downstream analysis follows as a later phase, with the advantage of starting from reliable, traceable data. The two can be commissioned in sequence, preparation first and then analysis, or separately if the client only needs the database put in order. Keeping the phases separate protects validity: an analysis of an unaudited database carries forward the issues this phase resolves.

In data quality and health database preparation, the client collects, owns and decides on the original data; the service audits, characterises, treats and documents. This applies to the whole service, because the boundary protects both the client and the validity of the work. The service identifies inconsistencies, assesses outliers and missingness, proposes and applies treatment strategies and documents everything. But decisions that depend on clinical knowledge or context, such as which value can be recovered, which record is valid or which inclusion criterion applies, are taken by whoever knows the data and where they come from. Imputation, exclusion and harmonisation are applied as justified, traceable techniques, not as silent changes to the client's database. The client keeps the raw database at all times, along with traceability to the analytical one. What you receive is a better-characterised, documented database with every decision in plain view, not an altered version you have to accept blindly.

We deliver the prepared analytical database together with the version-controlled preparation pipeline, the data dictionary, the log of data cleaning decisions and the traceability from the raw database to the final one. This applies to anyone who needs their later analysis to be defensible, because a result is only as auditable as the preparation behind it. Reproducibility matters for two concrete reasons. It lets any conclusion be traced back to the source data and the decision that treated them, and it lets the preparation be re-run on an updated version of the database without redoing the work by hand. The documented pipeline turns data cleaning into a reusable asset for the team, not a series of one-off adjustments that cannot be repeated. What you receive is not just a clean database, but the full trail of how it was reached, which is what makes it defensible to reviewers, committees or internal audits.