Back to Blog

Addressing the Challenges of Big Data in Healthcare


Addressing the Challenges of Big Data in Healthcare

As big data in healthcare continues its exponential growth, it represents new challenges of storage, access accuracy, and actionable use. Despite (or because of) the advance of HIT systems, the prevalence of data silos continue. Consequently, addressing these big data challenges requires careful planning for collection and processing while ensuring data integrity, so that it can be accessed and analyzed to reveal actionable outcomes.Many of the data challenges can be traced back to the vast amounts of unstructured data. According to a recent Stanford Medicine Health Trends Report, IDC predicts that an estimated 2,314 exabytes of health data will be produced in 2020. Most of that data will be unstructured.  This leads many healthcare enterprises to rely solely on structured data to guide their decisions. This can’t overcome the fact that even structured data can have quality issues.

Data Quality, Storage, and Integration

Healthcare organizations are constantly challenged to maintain the right level of data quality so decisions could be made for patient safety, improved clinical outcomes, and regulatory compliance. The ramifications of inaccurate data could impact patient safety, accurate reimbursement for services, and many other aspects of healthcare delivery. Data cleansing also identifies duplicate data, which directly affects the organization’s efficiency and effectiveness.

Download the Data Security Guide: Learn how to prevent, detect, and contain a  data security incident.

By implementing supplemental data cleansing tools, healthcare organizations can improve data quality before decisions are made or regulations compromised. Data cleansing is the identification and correction of corrupted, duplicate, missing, or inaccurate data.

Clean data will need to be stored before analysis or access, which requires future-proof, scalable, and cost-effective storage solutions. Healthcare enterprises are gravitating toward cloud object storage as a way to store massive amounts of data. Rather than organizing data in files or blocks, object storage can be increased for almost limitless scalability.

Object storage can be a major key to big data analytics through greater accessibility. Cloud-based solutions have emerged as a means of driving successful big data strategies. While the use of the public cloud can enable solutions for growing data storage needs, many organizations will adopt hybrid cloud structures with private clouds and multi-cloud environments as part of the mix.

These organizations can work with their private cloud provider to implement flashed-based arrays to take advantage of hyperconverged infrastructure  (HCI). Hyperconvergence virtualizes elements of datacenter infrastructure, including storage, networking, processing, and memory. In addition to greater visibility and control, HCI scalability reduces upfront costs while making it easier to store and access big data for analytics.

AI, Machine Learning, and Big Data

Currently, large patient data sets can hold numerous healthcare insights with the right tools to parse the data. AI and machine learning has become a viable way to facilitate data analytics for assisting diagnosis and treatment in a variety of ways that include:

  • Medical reconciliation to minimize prescription errors
  • Medical imaging and diagnostics to analyze chronic conditions
  • Leveraging lab and other medical data to enable early diagnosis

Just one of the clear and accessible ways that organizations use AI to extract meaning from big data has been through the IBM Watson Health™ cognitive computing system. Healthcare organizations are using machine learning to extract actionable insights from structured and unstructured data to solve pressing healthcare issues through data analytics.

Big Data Security, Compliance, and Governance

Big data in healthcare poses a number of challenges to security, compliance and governance that can lead to breaches. According to a recent edition of the HIPAA Journal, there were 86 healthcare data breaches reported in the fourth quarter of 2017 with more than a third  driven by improper insider access.

A comprehensive Identity Access Management (IAM) strategy is a critical means of ensuring regulated patient data access and regulatory compliance. Working in conjunction with federated single sign-on (SSO), in-house and third-party applications and adaptive multi-factor authentication (MFA) will allow healthcare enterprises to control authentication requirements based on the risk associated with the access request.

Healthcare organizations have access to many healthcare data management and integration solutions that can provide the needed data security, compliance, and governance. These solutions can do the following:

  • Connect to a variety of internal and external data sources
  • Seamlessly share clinical information and collaborate across the care continuum
  • Integrate structured and unstructured data from all sources
  • Deliver vendor-neutral solutions that are not bound to an EHR, HIE, or payer system
  • Provide support for open standards such as HL7, FHIR, IHE, SMART platforms, or SAML

While these data management tools enable greater security and compliance as well as governance, they must be used as part of a big data strategy that integrates both technology and policies. Many healthcare enterprises are turning to Managed Service Providers (MSPs) to help them design and manage appropriate data architectures. By partnering with the right MSP, healthcare enterprises can leverage critical IT skills and find the right tools to overcome the challenges of big data now and down the road.

Data Security Guide: Prevention, Detection, and Containment