Report on Cloud – Resource Efficient Data Analysis

There are several scenarios for one-time or periodic data analysis running on large data pools (such as predicting buying behavior, analyzing customer loyalty, retention management, early risk detection, …).

In addition to the technical requirements and dependencies, these mainly result in two challenges:

Challenge # 1: Performance-intensive Data Analysis = High Hardware Cost & Long Processing Time for Regular Analyses

Depending on the complexity of the data structure, analyses can have any runtime. Usually it is “too long”. To reduce this, large performance is required. But on which systems is this high performance available ? In most cases, the productive systems is intended for operation only, but not to do such resource-intensive analyses: the performance requirements of complex analyses would slow down productive operation.

Challenge # 2: The Handling of Personal Data (GDPR) = Customer and Employee Profiles.

Data Controllers are increasingly paying attention to the fact that sensitive, personal data may not be used for comprehensive profiling at the individual level without the consent of those affected. With the GDPR, there is a clear tightening in the regulations and especially with regard to the potential penalties for non-compliance. This raises the question: Give up data analysis or neutralize the data, even if the informative value suffers massively?

Do not compromise! Libelle’s cloud approach empowers you to address these challenges

Use dedicated systems exclusively for analysis and reporting purposes. Limit the number of users for these analysis systems. Avoid the investment in hardware, which runs most of the time with low utilization, but soonly is too weak for the purpose of analysis. Operate an analysis system that is regularly provided with fresh and meaningful (!) anonymized data.

Rely on analysis or reporting in the cloud. The advantage: the need-based provision of required resources and the usage-dependent billing. In concrete terms, this means that you operate – and pay – your cloud-based analysis and reporting systems

  1. only for those time windows in which analyses actually run.
  2. with exactly those resources (in particular CPU power and RAM) that are needed at the time.
  3. with anonymised but logically consistent data, which comply with both GDPR and the technical requirements.

Building and operating such systems is much easier than it is seen in many companies and departments. Libelle and BasisTeam support you with reliable best practices.

The Technical Planning

During technical planning, we usually work with your application, process, and data protection officers to define the optimal operating model. this includes

  • the definition of the specific requirements for the analysis system
  • based on the above, the pre-selection of possible cloud providers including PoC scheduling,
  • a decision template regarding the final cloud platform,
  • the detailed planning of the implementation and the operational phase,
  • the support of the implementation
  • and if necessary: support in the operating phase

The Implementation Project

After the technical planning, the implementation phase starts. Together with you we will build your analysis/reporting system in the cloud either “greenfield”, from scratch, or migrate / “lift & shift” an existing system to the cloud (Migrate2Cloud). In doing so, we rely on the extensive know-how of our consultants and on software solutions such as the LibelleDBShadow or LibelleSystemClone.

The Regular Operation

Regular operation usually runs in four main phases:

  • Normal Operation: The analysis system is used by the “regular” users for performance-neutral to simple requests and operated with a “sufficient” performance profile.
  • Performance Phase: With the announcement of large, performance-intensive analyses, the cloud system is inflated and operated with a “performance profile”. At the end of the performance phase, the performance profile is reset to normal operation.
  • Sleep Phase: In low- or non-working time, the performance profile is turned down to “minimal”, or the cloud system is completely decommissioned.
  • Refresh Phase: During the refresh phase, the current production data is transferred to the analysis system – in the case of SAP systems using Libelle SystemCopy, or in non-SAP applications via Libelle DBCopy. Here the process runs an with optimized performance profile, which increases or reduces, depending on the current task requirements. The last step of the Refresh Phase is to anonymize personal and other sensible data according to GDPR. Also in this step, the performance paramters are adopted to the specfic needs, before they are set back to Normal Operations or other profiles.

You have questions and would like to know more? Get in Contact.