GenAI-Assisted Training Needs Assessment for Public Administration: A Vendor-Independent Prompt Engineering Methodology

Traditional approaches to training resource allocation rely on top-down planning (PIAO - Piano Integrato di Attività e Organizzazione) without validation against actual employee skill gaps, resulting in misaligned training investments and unmeasured outcomes.

Unione Bassa Reggiana exemplified this challenge:

• Training needs determined by management intuition and ad-hoc individual requests
• No data-driven methodology to validate PIAO training priorities
• No baseline measurements to assess training effectiveness
• Limited resources (budget, staff time) for comprehensive needs assessment
• Uncertainty about GenAI tool applicability in PA contexts

The organization needed a cost-effective, scalable methodology to:

(1) systematically identify skill gaps,
(2) validate strategic training priorities,
(3) optimize limited training budgets, and
(4) establish measurable impact frameworks.

ER2Digit designed and facilitated a GenAI-assisted methodology for data-driven training needs assessment, explicitly engineered to avoid vendor lock-in through reusable prompt architecture.

Core Innovation: Vendor-Agnostic GenAI Integration

Rather than deploying proprietary solutions, the methodology leveraged publicly available LLMs (GPT-4o and Gemini 1.5, then 2.5 Pro) through structured prompt engineering.

The critical design principle was to craft prompts to be tool-independent, allowing seamless switching between GenAI platforms as models evolved or became unavailable.

E.g. system prompt example to counteract default LLM “sycophancy” and ensure critical analysis: “Be extremely accurate. Recommend things I’m unaware of that could benefit me. Be critical of my misconceptions. Be brutally honest, never servile. Be honest about my mistakes.”

Four-Phase Methodology (Plan-Do-Check-Act):

Phase 1 - Data Collection: GenAI-assisted questionnaire design aligned with PIAO requirements, collecting employee self-assessments across 6 competency dimensions (digital skills, regulatory knowledge, workload management, problem-solving, communication, collaboration).

Phase 2 - Intelligent Analysis: GenAI identifies critical skill gaps and maps them to operational risks and PIAO strategic areas. HR validates findings.

Phase 3 - Course Mapping: Prioritized needs mapped to publicly available training platforms (Syllabus, Accademia Comuni Digitali), ensuring 80% coverage.

Phase 4 - Impact Measurement: Post-training questionnaire re-administration (planned for future rollout).

The Pilot was implemented as about 10 HR Lepida employees tested the methodology to validate feasibility before full organizational deployment. ER2Digit provided methodological framework, facilitation, and staff training on GenAI prompt engineering.

Quantitative Outcomes (Pilot Phase) Process Validation:

• 100% satisfaction rate with methodology among participants
• 8 distinct skill gaps identified through structured analysis
• ~50 publicly available courses mapped to identified needs
• 80% coverage rate (needs matched to accessible training)
• Questionnaire completion time: less than 15 minutes per employee

Strategic Discovery:

About 20% of the PIAO training priorities validated by employee data Critical finding: the misalignment between planned training with actual workforce needs can prevent investment in low-priority training areas

Efficiency Gains (estimated):

Traditional needs assessment: estimated 40-60 staff hours (interviews, analysis) GenAI-assisted approach: less than 20 hours for questionnaire design, deployment, and analysis (i.e., 50-67% time reduction in planning phase)

Qualitative Benefits

• First-ever data-driven validation of training strategy
• Transparent, anonymous employee participation framework
• Measurable baseline established for future impact assessment
• Replicable methodology for annual PIAO cycles
• Systematic evidence to challenge misaligned top-down priorities
• Concrete course recommendations linked to identified gaps
• Reduced subjective decision-making in budget allocation

Technology Transfer:

• Internal staff trained on prompt engineering techniques
• Organization retained full methodology ownership (no proprietary dependencies)
• GenAI tool flexibility demonstrated (Gemini 1.5 became unavailable mid-project; seamless transition to 2.5 Pro)

DO’s

1. Engineer for vendor independence design GenAI integrations around reusable prompts, not specific platforms. Our methodology survived Gemini 1.5 discontinuation without disruption—a critical resilience factor for PA technology adoption.
2. Counteract default LLM behaviors: Public LLMs are optimized for user satisfaction (“sycophancy”), not critical analysis. Explicit system prompts demanding accuracy and constructive criticism are essential for professional PA applications.
3. Maintain human validation gates: GenAI identified skill gaps and suggested course mappings, but HR expertise validated findings against organizational reality. Human judgment remains irreplaceable for strategic decisions.
4. Quantify everything possible even in pilot phases, establish measurable baselines. Our 6-dimension self-assessment framework enables future pre/post comparisons that justify training investments to governing bodies.

DON'TS

1. Don’t ignore change management: “Moderate availability” of PSO staff was the primary constraint. Successful scaling requires: executive sponsorship, protected time for participation, and clear communication of anonymity guarantees.
2. Don’t over-rely on GenAI output: LLMs can hallucinate course recommendations or misinterpret context. Every GenAI suggestion underwent HR verification against actual platform catalogs.
3. Don’t skip impact measurement: Design Phase 4 (post-training reassessment) is non-negotiable for proving methodology value. PSOs must commit to multi-year measurement cycles, not one-time assessments.

Critical Success Factor: Prompt Engineering Training

• ER2Digit’s most valuable deliverable was teaching internal staff to craft effective prompts. This capability transfer ensures Unione Bassa Reggiana can replicate and refine the methodology independently—a requirement for sustainable PA innovation.