ggk-quote

Get A Quote

ggk-contact

+91 1234 44 4444

Data Management

Designed and implemented a suitable Hadoop architecture post analysis of various data sources.

Challenges

  • The client’s existing data warehouse was built on a RDBMS-based MPP system.
  • Data was being collected from various data sources like SAP ERP, JDA, IRI, Acousta and many others. The client was facing multiple challenges in integrating and performing real-time analytics with the data from these data sources.
  • The client wanted to switch to a scalable EDM system which is easy to integrate with other enterprise applications.
  • Wanted to minimize the effort spent on data migration and processing.

Solutions

  • Analyzed various data sources and targets, and designed and implemented a suitable Hadoop architecture for the same.
  • Designed near real-time ingestion system using Attunity and Kafka to perform real-time batch processing.
  • Designed a data warehouse on Hive and HBase for all transactional data processing.
  • Used Talend ETL to design and develop Spark streaming and batch jobs.
  • Designed OLAP cubes for reporting.

Tools & Technologies

Hadoop, Talend, SAP Hana, Kafta, Hortonworks, Hive

Key benefits

  • Production rate can now be matched with order rate, thereby enabling faster and more informed decisions for sales, pricing, product, and manufacturing teams.
  • Real-time operational visibility.
  • Quicker and better-informed decision making.
  • Lower data integration cost.
  • Easily scalable architecture.
  • Ability to drill across cubes/data subject in pre-defined and ad-hoc ways.
  • Attunity replicates the integrated data into Kafta streams thereby providing a comprehensive view to the managers.