How to be a good data program manager?
A spotlight on how to be or become a successful data program manager via data-centric behaviour.
A deep awareness of the data management life cycle, exceptional project management skills, and a solid grasp of benefits management is essential for effectively managing and executing large/complex data-centric programs. The road to success can be filled with potholes if such programs are managed without prior experience with data management, or if one assumes that a project manager without any data experience can successfully deliver a 'data program.'
Data-centric programs typically include:
- Data Platform Modernization — moving legacy / on-premises applications, workloads, data lakes, lake houses, data warehouses and data marts to cloud platforms.
- Building platforms/applications that rely on consolidating disparate types of data from multiple source systems and harmonizing them across dimensions.
- Data Lifecycle Management projects.
- Data-Centric Products.
- BI / Analytics, Dashboards and Reporting programs.
- Predictive Modeling / Data Science programs.
This article applies a spotlight on how to be or become a successful data program manager (DPM).
01Data Strategy and Data Governance
A Data Program Manager (DPM) should ensure that the program charter is aligned with the organizational data strategy and roadmap. Organizations with well-defined data governance standards and a clear roadmap see significantly higher success rates in their data programs. The data strategy, data maturity roadmap and governance policies should be referenced when writing or evaluating a data-centric charter. If the organization lacks the maturity to have a data strategy, the program is often the best opportunity to create one — the DPM should plan a small budget to build that strategy alongside the program (two birds, one shot).
02Architecture
Involve enterprise data architecture teams to review the project charter and certify the program's alignment to the laid-out strategy — this saves unpleasant surprises later. Assess the data volumes the program will create, distribute, ingest, process or store. That informs technical decisions, design, performance thresholds, environment sizing and other architectural choices. Most data-centric programs revolve around cloud platforms or cloud data warehouses, so a DPM should have a firm understanding of cloud offerings, data services, cloud DWHs, data lakes, data mesh concepts, DevOps and cloud cost calculations.
03Design
When defining or recommending a design, solution or data model, ensure your architects do not present a single choice to the decision board. For the best decision making, always present at least two options with a recommendation, so the board can evaluate against technical, schedule, budget, scalability, supportability and other short, near and long-term aspects.
04Discovery
Evaluate whether the program needs a preliminary discovery phase. Discovery uncovers unknowns hidden behind the scenes and minimises risks in development and testing. A discovery phase done before requirements enables bi-directional requirement gathering between the discovery team and end users.
05Data Integration
Data integration is key to the success of any data program. The DPM should have a thorough understanding of various data source types and the issues they bring, internal and external integration points and their limitations, ETL and transformation approaches, and the batch vs near-time vs real-time nature of the data integration needs.
06Data Quality
Poor data quality can derail a data program, so it is imperative to assess data quality in the context of the program. Data profiling should be conducted in the first stages to identify accuracy, completeness, consistency, relevance, comprehensiveness, granularity, homogeneity, redundancy and precision. A data-centric project will fail — or prove extremely unviable to re-engineer — if the underlying data is of low quality.
07Data Masking
When data masking is mandated, assess the risks it introduces to data integrity, master data and business rules. Testing with masked / obfuscated / scrambled data can lead to misleading results.
08Approaches and Standards
The program's quality plan should call out specific approaches, strategies and standards the program will adhere to — for example, standards for architecture, design, coding, documentation, error handling and DevOps.
09Methodology
Every DPM should have an excellent grip on project delivery methodologies. It is critical to understand how to design a program in agile to deliver measurable objectives and value, increase business visibility and enable quick rollouts. Agile is best suited for transformation programs where change is constant.
10Estimations
As a DPM, evaluate and have an extremely clear view of program estimates — especially if they were done by another person, team or entity. As you gain clarity, the first order of business is to assess whether the program can be delivered within the approved costs and estimates. Every inventory and estimation assumption should be at your fingertips to catch scope creep.
11Project Charter and Prioritization
Before starting a new program, it is the DPM's responsibility to know the program's priority within the portfolio so the right resources are allocated to deliver the desired value or outcome.
12Stakeholders
Stakeholder management gets neglected under the assumption that daily stand-ups, weekly status reports and periodic steering committees cover it. They do not — it is in the DPM's interest to have a deliberate stakeholder management approach.
Best Practices
- Spend time identifying stakeholders and the management style appropriate for each (refer to PMI for stakeholder management styles).
- Ensure at least 20% FTE involvement of a Business Owner / Business Change Manager so organizational and business changes are managed from start through handover.
- Ensure the program's business change manager is not new to the landscape and has the deep SME knowledge needed to influence the user community.
- Expect conflict and be prepared to deal with difficult stakeholders, unachievable expectations or tangential priorities — map out an influencer who can shape these stakeholders.
13Planning and Monitoring
Best Practices
- Produce a plan with high business and technical involvement; ensure the jargon used is clear and accepted by everyone. Walk through the Plan-on-a-Page (PoaP) periodically.
- Validate the assumptions the program started with — an unvalidated assumption can creep in and stall the program.
- Transformation programs involve unexpected changes. Build in progressive re-planning every week or fortnight, with a pre-defined threshold to accept changes.
- On long-duration programs, manage pace by juggling priorities so measurable output is delivered regularly. Avoid long SDLC phases at all costs.
- Do not create a single megaproject plan capturing the lowest level of detail. Multiple tracks, each with a detailed project plan owned by a track lead, work better; the program manager focuses on the rolled-up Plan-on-a-Page view.
- Most programs fail because risk and issue management is assumed to be the program manager's job alone. Empower teams to surface problems with mitigations.
14Collaboration
Best Practices
- Teams and stakeholders are increasingly remote — set up strong collaboration tooling for the program.
- Communicating with internal teams is as important as communicating with business stakeholders. Keep spontaneous comms to a minimum; plan all communication.
- Create a robust org structure with a clear chain of command, boundaries and authority. Leave no scope for interpretation; every team member is accountable for their responsibilities.
- Micromanagement demoralises your team. Empower them through trust and transparency.
- Delegate. A solid org structure frees up your time, prepares reportees for the next level, and keeps ~25% of your time available for unexpected issues.
15Change Management
Best Practices
- Define tolerances between decision-making authorities for scope, cost, time and quality.
- Do not just write 'changes are governed by change management procedures' in the program definition. Include a usable process and secure stakeholder buy-in to make it work.
16Testing
Best Practices
- Engage end-users from the beginning — design, development, testing and demo phases. Data programs need show-and-tell of current vs future state data structures to avoid surprises late in testing.
- Never assume defects will be caught and resolved in the testing phase. Most defects should be caught earlier (unit, system and integration testing with functionally viable data).
- Develop a quality process where engineers provide evidence of unit testing and hold dedicated sessions with BAs and architects to validate solution accuracy.
- Bake a clean test cycle into the overall plan — this cycle should begin with near-zero bugs, avoiding massive retest and fix efforts at the end.
- Conduct E2E testing as early as possible. This requires significant data preparation — never assume source data is readily available, it never is.
- Never neglect the infrastructure needed for development, SIT, E2E, performance, parallel and UAT tests with high volumes of viable data.
17Signoffs
Do not give up on getting deliverables signed off promptly. If approvals are not provided, try to obtain a conditional signoff so progress is not hampered by delays in formal sign-off.