Certified Reviewers

by
  • Role of a Certified Reviewer
  • Guidelines for accepting and conducting course reviews
  • How to use an individual assessment form
  • How to conduct a combined assessment
  • Combined Assessment Summary Report
  • Course accreditation
  • Your user dashboard
  • Email notifications

Role of a Certified Reviewer

A Certified Reviewer has completed the requirements to become an Interim Reviewer and had their role changed to that of a Certified Reviewer. Certified Reviewers assess courses assigned to them by the TELAS Administrator. They are allocated the role of First or Second Reviewer for any course assigned to them.

Guidelines for accepting and conducting course reviews

  1. Certified Reviewers are assigned course reviews by the TELAS Administrator based on the level of demand by participating institutions.
  2. The TELAS Administrator will assign courses to you by email as either a First Reviewer or Second Reviewer and you are asked to complete any assigned courses within a reasonable time frame. If you are unable to review an assigned course you should reply to the administrator as soon as possible so that an alternative reviewer may be assigned.
  3. You are deemed to have accepted an assignment unless you reply to your notification email and inform the TELAS Administrator that you are unable to accept the assignment. 
  4. An Admin Reviewer will always complete their part of a course assessment prior to it being assigned to you as a First Reviewer or Second Reviewer. You cannot edit the success indicators of a course reviewed by the Admin Reviewer. However, you are expected to select an Overall score for each standard in the assessment.
  5. Your completed assessments are displayed in your user dashboard.
  6. After completing your individual assessment of a course, you will be expected to undertake a Combined Assessment with the other reviewer assigned to the course but only after both of you have completed your individual assessments.
  7. If you have been assigned a course as a First Reviewer, you will complete your review before the course is assigned to a Second Reviewer. Once you have both completed your individual assessments, you will be asked to complete a Combined Review.
  8. If you are designated as the First Reviewer, it means you are the Lead Reviewer and will be expected to initiate a Combined Assessment by contacting your Second Reviewer. 
  9. If you are the First Reviewer, you will be provided with the contact details of the Second Reviewer once you have both completed your individual assessments.
  10. Before commencing an assessment you will need to call or email the Course Enrollment Contact in order to make arrangement to enroll in the course. You will find these contact details above the assessment form in the Course Details Panel.

How to use an individual course assessment form

  1. The assessment form is divided into separate pages to make the screen layout of forms more user friendly. View this diagram for a description of the different elements in the assessment form layout.
  2. Assessment forms contain the following hierarchy of elements:
    – Standards: There are 8 standards in the form – two for each of the four domains.
    – Performance Criteria: Each standard contains a number of performance criteria.
    – Success indicators: Each performance criteria contains a number of success indicators.
    – Measure of Performance: the score for each success indicator.
    – Overall Score: the overall score for each performance criteria.
    – Comments: available at the end of each standard.
  3. When you open an assessment form, you will find a Course Details Summary panel displayed at the top of the page displaying critical information about the course.
  4. As you can see from the diagram, each page on the assessment form displays several Performance Criteria. The Success Indicators for each Performance Criteria are displayed by selecting the dropdown arrow visible on the left of each Performance Criteria.
  5. You score each Success Indicator by selecting one of four choices – Yes, Yes But, No But or No. Selected scores are colour coded when clicked.
  6. You must also select the Overall Score for each Performance Criteria. The Overall Score is a summary score you assign to each Performance Criteria and its Success Indicators. For example, if you selected Yes, Yes and No for the three Success Indicators, you may decide to assign Yes But or Yes to the Overall Score.
  7. Admin Reviewer scores are displayed in dark grey to differentiate them from your own scores which are marked in blue. You cannot edit the Admin Reviewer’s individual scores but you are responsible for assessing the Overall Scores for those Performance Criteria scored by the Admin Reviewer.
  8. The Form has a progress bar above the Standards row to indicate which standard you have completed and how far you have progressed in the assessment.
  9. The Standards row also displays the percentage completed on the far right. If the % indicated is less than 100% at the end of your assessment, it means you have missed one or more scores somewhere in your assessment. Missed scores are identified by clicking each of the Standards on the Standards row until you locate an underlined Performance Criteria description. Dropdown the underlined Performance Criteria and you will see which Success Indicators have not been marked. Mark these and you should then see 100% displayed on the Standards row with a Mark Complete button available below your assessment.
  10. A comment field is available at the end of each standard in the form.  A prefix user type identifier is automatically added to each comment once the assessment is saved, i.e. [Admin-Reviewer], [First Reviewer] or [Second Reviewer]. Do not add your name to any comment but do include a reference number as to which success indicator your comment applies.
  11. Assessments can be saved at any point and you are able to return later to complete the assessment.  Once complete, you must select Mark Complete.

How to conduct a Combined Assessment

The Combined Assessment is undertaken by both the First and Second Reviewer in an online meeting or face-to-face. In most cases, these meetings will be conducted online using separate workstations with both reviewers logged in to TELAS. The First Reviewer is responsible for entering scores for the Combined Assessment in consultation with the Second Reviewer during the online meeting.

  1. First Reviewer role:  From your dashboard, select Combined Reviews -> Assigned -> Course Title -> Combined Assessment.  This view allows you to enter new scores agreed to by you and your second reviewer during your meeting.
  2. Second Reviewer role: From your dashboard, select Combined Reviews -> Comparative -> Course Title -> Comparative Assessment.   This view displays both the First and Second Reviewer individual scores side by side for comparison and reference purposes. It is only displayed in user dashboards after the First and Second Reviewers have both completed their individual assessments.
  3. Self-Assessment results: Both or you will be emailed a copy of the institution’s Self-Assessment results for the course and which you both should compare to your own results. You are asked to comment on any significant difference between your individual assessments and the results of the institution’s assessment. Note that the institution’s self-assessment will have been conducted using a hard copy of the TELAS Framework.
  4. Suggested workflow: Although both reviewers have access to the Comparative Assessment, it may be more convenient for the Second Reviewer to view the Comparative Assessment while the First Reviewer views and enters the new scores in the Combined Assessment. Using this approach, the Second Reviewer can identify and flag differences in their respective scores, discuss these with the First Reviewer and agree on the Combined Assessment score – which the First Reviewer then inputs in the Combined Assessment form.
  5. Closing an assessment: Once Standard 8 is displayed in the combined assessment form, you will see three options – Pending, Save and Mark Complete. Save is selected if the assessment is incomplete and a subsequent meeting is required to continue with the assessment. Mark Complete generates the Combined Assessment Summary Report showing the level of eligibility for course accreditation and indicates you have finished the assessment. Pending is selected in the event that the First and Second Reviewers fail to agree on one or more success indicators. View a sample of the report here.
  6. Pending assessments: The course assessment Pending option is selected when the assessment is intentionally incomplete and must be referred to the TELAS Administrator because the First and Second Reviewers have failed to agree on the scores for one or more success indicators. If this happens, the First Reviewer must do three things prior to saving the assessment:
    – Select Unmark for any success indicators that are in question.
    – Add a comment below the relevant Performance Criteria describing your different positions as reviewers regarding those indicator(s) in question.
    – Select Pending to save and close the assessment which will then be referred to a third reviewer to score the blank Success Indicator(s) by taking into consideration the First and Second Reviewers individual assessments and comments.
  7. Summary Reports: Once a Combined Assessment is completed and saved, the First and Second Reviewers will find a Combined Assessment Summary Report in their respective dashboards under Combined Reviews -> Comparative -> Course Title -> Summaries. The Summary Report is also emailed to the institution’s Course Submitter and this report signals the end of the course assessment process, with the exception of the TELAS Administrator issuing a Course Accreditation Certificate.

Combined Assessment Summary Report

  1. A Combined Assessment Summary Report is generated for every completed Combined Assessment and is made available to the First and Second Reviewers in their respective dashboards under Combined Reviews -> Summaries.
  2. The Combined Assessment Summary Report displays the results of the Combined Assessment and includes the Course Details, the eligible badges for each domain and the overall eligible badge for the course.  It does not contain the entire set of scores at the success indicator level. This data is available to institutions as an optional report.
  3. Summary data is emailed to the Course Submitter via the TELAS email notification system.
  4. A Summary Report is not generated when the First and Second Reviewers are unable to agree on one or more scores during a Combined Assessment.  In this case, the course assessment is referred to the TELAS Administrator and to a third reviewer for completion.
  5. If a course fails to meet the minimum score for badge eligibility, the institution has 8 weeks to evaluate the results of the course review, make changes to the course and then resubmit the course for a second review.

Course Accreditation

  1. A course is accredited if the Course Submitter opted for formal assessment at the time of submitting the course and when the overall course score in a Combined Assessment meets the minimum level required for course accreditation.
  2. The TELAS Administrator issues a Course Accreditation Certificate and badge to the institution via the Course Submitter if the above conditions are met.

Your user dashboard

  1. As a Certified Reviewer, you have access to a user dashboard that lists any and all courses they have been assigned to you along with the status of each course assessment.
  2. Your user dashboard is only populated when a course assessment has been commenced.
  3. Your individual course assessments are accessed from the Assessments tab in the sidebar under either Assigned, In-Progress or Completed depending on the stage you are at in the assessment.
  4. New courses that have been assigned to you are accessed from the sidebar under Assessments -> Assigned.
  5. Course assessments that you have partially completed are accessed from the sidebar under Assessments -> In-Progress.
  6. Course Assessments that you have completed are accessed from the sidebar under Assessments -> Completed.
  7. The Combined Review tab has six tabs:
    – Assigned, In-Progress and Completed tabs that reflect the status of each Combined Assessment.
    – Comparative tab that displays the First and Second Reviewer course assessment scores side by side. Referred to during a Combined Assessment to compare the two reviewer scores but only made available once the First and Second Reviewers have both completed their respective assessments.
    – Summaries tab displays Combined Assessment Summary Reports with eligible badges.
    – Accredited tab displays a list of the courses from your institution that have been assessed and accredited.

Email notifications

  1. Certain actions by users in TELAS trigger auto generated emails to users.   Many of these emails contain details of assigned courses and instructions.
  2. As a Certified Reviewer, you can expect to receive the following emails:
    • An email to confirm your registration.
    • An email whenever you are assigned a course to assess.
    • If you are the First Reviewer, you will receive an email asking you to contact the Second Reviewer to arrange an online meeting to conduct a Combined Review. If you are the Second Reviewer, you will be contacted by the First Reviewer once you have completed your individual assessment.
  3. Emails are sent to the email you submitted in your User Profile. 
  4. All emails to users are sent from admin@telas.edu.au and subject lines are preceded by the word [TELAS].