Online Standardisation tutorial

Transcript

Introduction

Hello, and welcome to this AQA tutorial for online standardisation.

The tutorial is approximately 20 minutes long and can be viewed as many times as you wish.

In addition to this tutorial, there is also a system preview component within the software which will allow you to view the online standardisation screens before you come to use them. There is also a comprehensive user guide which you can access and print from links within the software, as well as a set of frequently asked questions for reference.

If you need further help with any aspect of online standardisation the AQA Contact Centre will be available from the 28 April between 8am and 9pm on weekdays, and from 11am and 3pm at weekends and bank holidays. The number to call is 0870 410 1035.

AQA's online standardisation software has been developed in collaboration with a technology partner called Krankikom.

The software uses leading edge Web 2.0 and Adobe Flash and Flex tools, and is delivered over the internet.

This means that as an examiner with AQA, you complete your standardisation and in some cases your first-phase sample too online:

So

  • from a place of your choosing
  • at a time that is convenient for you, within a set standardisation period, and at your own pace
  • you can choose a screen layout that you prefer and you can print off a copy of the Mark Scheme
  • you can mark question by question, script by script or a combination of both
  • you'll receive immediate feedback on your standardisation marking which you can also refer back to later
  • and you can engage in electronic dialogue with your senior examiner using the messaging system.

For AQA, online standardisation offers a number of additional advantages over traditional standardisation.

Common scripts are used for online standardisation and for something in components not using CMI+.

Consequently AQA can:

  • make sure that examiners are given consistent information, training and feedback
  • can monitor examiner performance based on objective data
  • can use team leaders time more effectively. Time previously spent on over-marking live samples can now be spent interacting with and mentoring examiners and providing quality feedback
  • AQA can also cut down the time needed for script marking by eliminating delays resulting from the postal exchange of scripts.

The software is easy to use and we look forward to working with you in introducing improvements in our standardisation processes.

Before you begin

Before you begin please read this advice and then adjust your chair and your PC so that you are able to work comfortably.

The Standardisation process

Through the standardisation process, examiners learn to apply the marking standards outlined in the Mark Scheme correctly and consistently.

For both traditional and online standardisation, there is a pre-standardisation meeting where the principal examiner and his or her team of team leaders refine the mark scheme, taking into account how candidates have answered the questions on the question paper. The senior team also selects scripts for the standardisation of examiners and agrees marks for them.

For non CMI+ components the senior team will also select scripts and agree marks for the first-phase sample. Please note that throughout this tutorial further references to sampling do not apply to components using CMI+.

Traditionally examiners attend a standardisation meeting where the principal examiner explains the mark scheme, and examiners and team leaders mark scripts together in teams.

For online standardisation, examiners connect to the AQA online standardisation website via the internet. The software provides a series of exercises that guide examiners through the mark scheme, show how the mark scheme is applied in the marking of a model marked script, allow examiners to mark standardisation scripts, receive immediate feedback from the system, and mark and submit their first case sample to their team leader.

Traditionally team leaders have sent their examiners feedback on the first-phase sample forms. For online standardisation team leaders will clear examiners to mark and provide feedback by the systems messaging feature and by the telephone.

The online standardisation process

The online standardisation process comprises the following tasks that have to be completed in a pre-set order:

  • First of all, connect to the AQA online standardisation website and log in
  • select a component to mark or choose the system preview component
  • read the principal examiner's welcome page
  • read the examiner handbook and briefing
  • assimilate the question paper and mark scheme
  • and, if not using CMI+, use the model marked script to learn how to apply the mark scheme
  • then mark standardisation scripts and read the immediate feedback from the system
  • mark the common first-phase sample scripts and send your marks to your team leader electronically
  • review the online feedback
  • your team leader will also provide feedback by telephone and may ask you to select and mark an additional first-phase sample from your own allocation
  • when you're cleared to mark you should begin to mark your live scripts
  • halfway through the marking period, mark fifty second-phase sample scripts from your own allocation and send them to your team leader in the usual way
  • when you have finished all your marking send your marked scripts to AQA.

Getting started on the Welcome screen

Once you have logged in, the system will recognise your personal details.

If you're marking more than one component, a page will be displayed listing the components relevant to you and the system preview component.

Click on a system preview component to view all the online standardisation screens that will become available to you (this screen lists just the test components only).

Click the name of the component in which you want to standardise your marking.

The Welcome screen will be displayed, and will show the principal examiner and a welcome message. All the screens are set up in the same way, so that at the top of the frame there is a blue bar. On the top left, there is a task list which displays the current task and the date the task will be accessible.

When you click on the down arrow, a dropdown menu will appear.

This displays all the tasks and the sequence that you have to work through them to complete the standardisation process. As you complete a task, the next task in the process becomes active and clickable.

Click on the down arrow again to fold the task list in.

On the far right, the blue bar shows the component that you are standardising, the due date when all the tasks must be completed and your progress through the tasks.

The blue bar also shows your details and welcomes you personally. There is a Log out option and there will be a change component option. If you're marking more than one component you can move between components without having to log out each time.

On the far right and below the blue bar, the help desk number is displayed.

The messages tab alongside shows how many messages there are in your inbox. Click on the envelope icon to open the messaging facility and use this to send messages to, and receive messages from your team leader.

The information and facilities at the top of the frame are displayed at all times. During the standardisation process the principal examiner may publish new versions of important documents, such as mark schemes.

The system will send an update alert like this, so that you are aware of any changes. All tasks will be inactivated until you acknowledge that you have read the important notice by clicking ‘OK’ in the bottom right of the box.

Examiner Handbook and Admin briefing

When you have read the welcome message (task 1), you can move to task 2.

Click the down arrow to unfold the task list. Click on task 2: The Examiner Handbook and Admin briefing and the relevant screen will open below.

Click on the word ‘here’ to open the document.

You can then print the document out if you wish.

Once you have read the Examiner Handbook and Admin briefing confirm this by clicking on the check box here and then click ‘submit’.

The progress bar in the top right will show that you have completed 15% of the tasks required for full standardisation.

When you have finished task 2 and when the question paper and mark scheme have been published task 3 becomes active and clickable.

Navigating between questions in tasks 3-6

I'll now explain navigating between questions and scripts in tasks 3 to 6.

There are 3 ways of moving between questions and scripts, and you can select whichever method you prefer or, use a combination of all three.

Firstly from the overview page click on the box alongside the question number, this will take you to the relevant question or script and corresponding section of the mark scheme.

Alternatively, you can click on the task tab, where you can move to boxes on the left, and navigate between the different questions – and to return click again.

From there you can click the ‘Next’ button to access the next question. You can choose to mark question by question, script by script, or to use a combination of these modes.

The general functionality of the screens for tasks 3 to 6 is broadly similar too.

If a clip is too long to be displayed on the screen, a scroll bar appears on the right, which can be used to scroll the page up and down.

The zoom sliders on the left can be used to enlarge or reduce the size of the clip and of the mark scheme.

Screen layout can be altered by clicking on the screen layout icons in the top right-hand corner. There is a top/bottom layout, a side by side layout, or a single view screen which you may prefer if you have a printed out copy of the mark scheme.

The size of the panes can be enlarged or decreased by clicking on the dividing bar and dragging it until the layout suits your needs.

Items can be printed where there is a printer icon, here in the top right; so this indicates that the question paper in task 3 can be printed. Similarly, the mark scheme in task 3 can also be printed.

When you are ready to complete your standardisation marking, select task 5 from the task list.

Select a script from either the overview screen or the standardisation tab.

Select the first script, select question to mark and then enter marks in the script; click anywhere in the window displaying the script and a mark dialogue box will appear.

Select the mark type as explained in the mark scheme, and then enter the mark that you want to award, click ‘OK’ and the mark will appear in the body of the script and also in the margin on the right.

To add an additional mark, click again. The dialogue box will appear; enter the mark value and the mark type, ‘OK’ and then you can see that those two marks have been summed in the margin.

To change a mark, click again click ‘Delete’, and the mark has now been changed.

When you have completed marking a question, click on the lsquo;submitrsquo; button.

Once you have submitted your marks, the system will immediately display the correct marks in green alongside your mark in red.

You can then continue to mark the next question in script 1 or mark the same question in script 2.

As you progress through your standardisation marking the overview screen shows the questions that you have already marked and questions awaiting marking.

The white boxes show questions that have not been marked.

Green ticks show that a correct mark has been awarded.

Split boxes show the correct mark at the bottom with a green background, and your mark, as examiner is shown above.

An amber background indicates that the mark difference is within tolerance, and a red background indicates that that mark difference is outside tolerance.

When you have finished all the standardisation scripts, return to the overview screen, check the box that says I have completed all the marking and then press submit. The system provides immediate feedback. Additionally though you may wish to discuss mark differences with your team leader.

Non CMI+ examiners can now move on to task 6, which is the first-phase sample.

For CMI+ examiners your team leader will confirm that you are cleared to mark by telephone or by using the messaging facility. Your team leader will also confirm this with AQA so that the staff can switch the CMI+ application from training to marking mode.

Marking the first-phase sample

For non CMI+ examiners the marking of the first-phase sample works in the same way as marking of the standardisation scripts. Similarly, once you have submitted your marks to your team leader, the system provides immediate feedback.

Your team leader will contact you to provide feedback as well and either clear you to mark your live scripts or, to ask you to mark an additional first-phase sample from your live script allocation.

Please remember to use all the support available to you and if you have any further queries telephone the AQA contact centre on 0870 410 1035, or e-mail help@aqa.org.uk.

Thank you.

back