4.1 OVERVIEW
4.1.6 Quality support and assessment system
The Quality Support Group
The Quality Support Group (QSG) has been established to oversee the ongoing development and management of the quality support and assessment process, its objectives being to:
• Support improvements in the quality of programme/project ideas and documentation;
• Ensure screening is carried out in a harmonised way using a set of consistent quality criteria and standards;
• Ensure appropriate reporting and follow-up; and
• Identify and exchange best practices and innovative approaches.
The QSG undertakes screening of proposals and provides advice to line-managers at two key points in the project cycle:
(i) at the end of the Identification stage _ when the Identification Fiche (individual projects) or the Financing Proposal (for a Programme of projects) is reviewed; and
(ii) at the end of the Formulation stage _ when the draft Financing Proposal has been prepared (for individual projects) or when further specification of the Technical and Administrative Provisions has been undertaken (for projects under a Pro-gramme).
The QSG operates formally at two main levels, namely at the Office and Directorate levels. The respective composition and responsibilities of the Office and Directorate Quality Support Groups are shown in the table below.
In practice, quality support and assessment work must also generally be undertaken (indeed start) at Delegation level through a process of ‘Peer Reviews’.
Further information on the work of the QSG can be found on the intranet at:
X:/Europeaid/Thematicnetworks/qsg/Home_Page_
QSG_en.htm
Level Composition and tasks
QSG at the Office level Composition: Deputy DG, Head of Coordination Unit, Relex/Dev
Secretariat: Unit O3
Tasks: Establishing minimum standards, procedures, assessment criteria, review of key issues of interest to the office, synthesis of findings, reporting on findings
QSG at the Directorate level Composition: Director, HoU Coordination, qualified officials Tasks: Peer review of proposals (Identification Fiches and Financing Proposals), recommendations for improving the
The Quality Frame
At each main decision point within the project management cycle, a set of quality assessment criteria are provided to support structured and consistent analysis and decision making. These criteria are also reflected in the information requirements of key documents such as the Identification Fiche and the Financing Proposal.
While a common overall framework for quality assessment has been established (the Quality Frame),
it is recognized that different circumstances will require that specific issues are analysed in more or less detail. Line managers and others involved in applying the criteria and standards must therefore apply their judgment and experience to determine which are more or less important in relation to the context within which they are working.
The Quality Frame is shown in Figure 6 below:
17Standards listed under this criterion may not be directly applicable to projects implemented through non-governmental partners.
A
Consistent with, and supportive of, EC development and cooperation policies Consistent with, and supportive of, Partner Government policies and relevant sector programmes17 Key stakeholders and target groups are clearly identified, equity and institutional capacity issues analysed, and local ownership demonstrated Problems have been appropriately analysedLessons learned from experience and linkages with other ongoing/planned
The project is well designed and will deliver sustainable benefits
to target groups The objectives (Overall objective, purpose and results) and the work programme (activities) are clear and logical, and address clearly identified needs
The resource and cost implications are clear, the project is financially viable and has a positive economic return Coordination, management and financing arrangements are clear and support institutional strengthening and local ownership
The monitoring and evaluation (M&E) system and audit arrangements are clear and practical
Assumptions/Risks are identified and appropriate risk management arrangements are in place The project is environmentally, technically and socially sound and sustainable
The project is delivering the anticipated benefits and
is being well managed The project remains relevant and feasible
Project objectives are being achieved The project is being well managed by those directly responsible for implementation
Sustainability issues are being effectively addressed
Good practice principles of project management are applied by EC Task Managers
Figure 6 – Quality Frame
As can be seen, the quality frame consists of three key quality attributes, namely:
Under each of these 3 main quality attributes are a number of key criteria (total of 16), which indicate the key issues that need to be assessed in order to make a judgment about quality. Each of the key criteria is then supported by a set of key quality standards, which aim to provide further guidance with respect to promoting quality. These standards are presented within the text of Sections 4.3 to 4.5 of the Guidelines, which respectively deal with each main stage of the project cycle.
The key factors affecting sustainability of project benefit streams, as identified by the Development Assistance Committee18, are integrated within this Quality Frame, rather than being presented as a separate set of criteria.
The quality criteria and standards can be applied/
used at three main levels, namely:
• By Task/project managers (at Delegations and at HQ), in the course of their ongoing project cycle management responsibilities;
• By technical and other support staff (i.e thematic networks, QSG, consultant TA) in the course of providing technical support or assessment services to EuropeAID line management; and
• By line-management, in helping them to assess the overall quality of EuropeAID’s
programme/project portfolio in terms of quality of design, achievement of objectives and quality of project management.
The Quality Frame can either be applied with or without the use of a rating scale for each of the key criteria and standards. Use of a rating scale is a valuable tool if one wishes to make comparisons between projects, or if the intention is to gain an overview of the quality of a large portfolio of projects over time.
Link between the quality frame and the EC’s evaluation and audit criteria
The five criteria used by the EC to evaluate the success of projects or programmes are: (i) relevance; (ii) efficiency; (iii) effectiveness; (iv) impact and (v) sustainability (see section 4.6.4). The two criteria used by the EC to audit the performance of projects or programmes are: (i) efficiency (including economy); and (ii) effectiveness (see section 4.7.4).
In the context of the quality frame’s key quality attributes, the word Feasibility is used to describe the expected efficiency, effectiveness and impact of the project prior to the start of implementation. Effective and well managed describes the actual efficiency and effectiveness of the project during implementation, while the issue of impact can only be assessed through ex-post evaluation. As already noted, the factors promoting sustainability are threaded throughout the attributes of Relevance, Feasibility and Effective &
Well-managed.
18(i) Ownership by beneficiaries, (ii) policy support, (iii) appropriate technology, (iv) environment, (v) socio-cultural issues, (vi) gender equity, (vii) institutional and management capacity, and (viii) economic and financial viability.
• Relevant – the project meets demonstrated and high priority needs
• Feasible – the project is well designed and will provide sustainable benefits to target groups
• Effective and well managed – the project is delivering the anticipated benefits and is being well managed
Attributes x 3
Criteria x 16
Standards x 64
Increased detail