18 _ Validation policy and procedure

Purpose of the policy

This policy and procedure outlines Allara Learning’s approach to the validation of its assessment system, practices and judgements made by assessors.

It includes guidance on the 5-year validation plan, which describes who will lead and participate in the validation activities, and a validation schedule for each training product on the scope of registration.

Further, it outlines how recommendations for improvement to the assessment tool, process and or assessment outcome arising from validation assessment outcomes will be acted upon.

Pre-validation

Pre-validation team

Prior to assessment tools being implemented within the assessment system, the General Manager will convene a pre-validation team to conduct pre-validation of assessment tools.

The pre-validation team may consist of:

Trainers and assessors directly responsible for delivering the training product,
Other trainers and assessors,
Independent VET consultants,
Industry stakeholders and employers.

Pre-validation and outcomes

The pre-validation team is to review assessment tools to ensure that:

the assessment is consistent with the requirements of the training product; and
assessment can be conducted in a way that is consistent with the principles of assessment; and
assessment can be conducted in a way that is consistent with the rules of evidence

A report on the outcome of pre-validation, identifying the assessment tools compliance with the unit requirements, principles of assessment and rules of evidence is to be documented and distributed to the General Manager and Learning & Development designer for actioning.

Identified gaps are recorded in the Product development schedule.

Revised assessments

Assessment tools are revised in alignment with the guidance provided by the pre-validation team.

Revised assessments are further reviewed by the pre-validation team and if required a subsequent report of gaps or improvements is provided.

Approved assessment tools

A record of approved assessment tools is made within the Training Product and/or Training Resource Register, and copies of all training and assessment materials are stored and made available for access and use within the Training Products Library.

Validation of assessment

The validation team

The validation team will consist of one or more people who collectively have:

industry competencies, skills and knowledge relevant to the training product;
a practical understanding of current industry practices relevant to the training product; and
one of the credentials for validation specified in the Credential Policy.

Note: An individual may hold one or more of the requirements listed above.

The General Manager will convene and appoint a lead validator. The lead validator may appoint one or more people as a member of the validation team.

Credentials for all members of the validation team will be collected and stored in the validation schedule (see below).

Validation schedule

Each training product is to be validated at least once every five years, with high-risk training products will be evaluated on a minimum 24-month cycle from the date of their original release.

Allara Learning will maintain a validation schedule reflecting the current scope of registration. The schedule is a five-year plan for the validation of training products that sets out:

The training product to be validated,
The individual units of competency and/or subjects to be validated,
The training products identified risk level,
The assigned validation team and their respective credentials,
The year and month validation will occur,
The due date for validation outcome reports,
The status of the validation outcome, and
A high-level summary of the validation outcome.

When a new training product is added to the schedule, it will be adjusted to ensure it continues to meet the validation timing requirements.

Risk rating of training products

High-risk Training Products will be validated on a 24-month cycle as opposed the standard 5-year validation cycle. High-risk training products are defined as having any of the following characteristics:

Training products identified by ASQA as high risk;
Licensed/Regulated Courses:
- Responsible Service of Alcohol (RSA); and
- Responsible Conduct of Gambling (RCG); and
- Food Safety Supervisor (FSS).

Medium-risk training products are validated at minimum every 5 years, however, may be validated more often in response to feedback from students, employers, and industry. Medium-risk training products are identified by the volume of annual enrolments in the training product.

Low-risk training products are validated every 5 years. Low-risk products are identified as all other training products with low annual enrolments.

Order of validations

The order of validation activities listed within the Validation Schedule will be determined by a hierarchy of selection:

High risk products,
Highest volume of enrolments in a qualification,
Highest volume of enrolments in a unit of competency,
Requirement to address changes in technology or workplace processes.

Validation components and sample size

The components of the assessment system and sample size of assessments to be validated will be determined by:

The identified risk rating for the training product, and
The volume of past and current enrolments in the training product.

Sample assessments for the sample size must be a completed via a random selection process. The following are the steps required to select a random sample:

Identify trainers/assessors that have delivered the training product
For each trainer/assessor identify the different employers (apprenticeship or traineeship)/program (Jobseekers)
For the above, produce an alphabetical listing, sorted by surname of the students whose work was assessed and sample at least one from every 5th letter until the sample size requirement is met

Where RPL is used for the relevant unit, and no RPL completions have been gathered in the sample, additional RPL samples will be added to the assessment judgments.

Note: A minimum of two (2) individual units of competency must be sampled when validating a full qualification training product.

Effective validation

Validation must only consider the assessment evidence collected, retained and recorded in support of an assessment judgement for the selected assessment samples.

Validation determines if assessment tools have produced the intended evidence. Validators must look at the evidence in the sample, and determine if:

assessment has been conducted in accordance with the principles of assessment (fairness, flexibility, validity, reliability); and
judgements have been made in accordance with the rules of evidence (validity, sufficiency. Authenticity, currency); and
Evidence collected demonstrates consistency with the requirements of the training product; and
Judgements made are comparative and consistent across multiple assessors.

Further, validators are to review assessment instruments to ensure they:

Are appropriate to the contexts and conditions of assessment (this may include considering whether the assessment reflects real work-based contexts and meets industry requirements)
Are appropriate in terms of the level of difficulty of the tasks to be performed in relation to the skills and knowledge requirements of the unit
Provide sufficient instruction to clearly explain the tasks to be administered to the learner (if the assessment samples demonstrate the evidence provided by each learner is markedly different, this may indicate that the instructions are not clear)
Give sufficient guidance as to the evidence to be gathered from the learner
Provide sufficient instructions for the assessor on collecting evidence, making a judgement, and recording the outcomes of the assessment (assessment samples should validate recording and reporting processes)
Outline appropriate reasonable adjustments that could be made to the gathering of assessment evidence
Are supported with evidence criteria (benchmarks) to judge the quality of performance.

Validation outcomes

Throughout the validation cycle, the observations made, and outcomes of the validation will be documented within a validation report.

If deficiencies are identified during validation, these will be highlighted within the validation report.

Final outcomes and recommendations for improvements are recorded in the quality assurance ‘Validation schedule’.

Rectifications and amendments

The highlighted deficiencies and recommendations required to achieve compliance with the relative unit/s of competency and training package will be added to the ‘Continuous Improvement (CI) action items register’.

A member of the Learning & Development team is assigned the action item to implement the recommended changes.

When all amendments have been made and reviewed in accordance with the Pre-validation process in this policy, the following occurs:

The updated assessment tolls are added to the relevant Training Product and/or Training Resource Registers
Copies of all revised assessment documentation are stored and made available for access and use within the Training Products Library.
A record of the changes made, and release of the updated resources is recorded in the Continuous Improvement (CI) Register.

Records Management

Allara Learning’s records management processes are used to retain the following validation evidence:

The person leading and those participating in the validation activities (including their qualifications, skills and knowledge);
The report of student outcomes used to define the sample size;
The validation tools used;
All assessment samples considered;
The validation report and outcomes; and
Any rectification required.