Case Study Computer Based Examination System In Ancient

Computer-based exams and other thoughts.

How many times have you heard someone at work talk about “the good old days” and how “exams were harder in my day”. Whether it’s GCSEs, A levels or accountancy exams, this has been an ongoing conversation for many years.

The latest water-cooler topic is computer-based exams (CBEs) and how they compare to the “traditional” handwritten papers. If you are studying towards the CIMA, ACCA or ICAEW exams, for example, at least some of your exams will have to be taken on a computer, with “objective test questions” and multiple-choice becoming commonplace.

Paper-based woes

Let’s look at the facts – no system of examination is perfect. The traditional paper-based can test different areas of the syllabus, often in rotation, so that over a year, most core topics are examined and some aren’t. This means that you can “get lucky” or unlucky if a topic that you are good or bad at appears or doesn’t appear in a particular paper.

CBE pros

With CBEs, more of the syllabus can be tested so that “question-spotting” becomes irrelevant and any weak areas that a student has are exposed.

One question can test several different rules. You have to know your stuff and then think, really think, about what you have to do to solve the problem. Get one aspect of a question wrong and all the marks go down the plughole.

How are today’s exam candidates affected?

These days you have to know your stuff, across a much wider range of topics, and be able to apply your knowledge to some very specific situations. Anything less than precise, detailed, knowledge will result in no marks being scored.

In my view at least, this is a much tougher ask than in the past.

CIMA exams

In the CIMA case study exams, now examined throughout the qualification rather than just at the end, candidates are receiving a very good general business education right from the outset, and developing communication and judgement skills to complement this education. I have been very impressed with some of the new breed of candidates who make it through to the strategic case.

Overall then a win win for CIMA students and their employers. Precise, targeted, knowledge orientated multiple choice exams, that can be taken on demand, and in depth investigations into three different industries across the qualification.

ACCA exams

Now, and in the interests of remaining reasonably neutral, I got a huge kick out of teaching the ACCA paper F6 , taxation, for the December sitting. It is a mix of multiple choice, short form scenario based questions and longer questions. In my view the new format is a big improvement on the old approach in that a broader range of topics can be examined in any one exam.

ICAEW exams

I am also looking forward to the challenge of preparing students for the new ICAEW computer based exams. The ICAEW have done a great job with their approach to examining accounting on the computer and I am sure they will get my other papers right!

Looking to the future

It is fantastic to see how the professional body exam boards are evolving to meet the needs of today’s organisations.

The future looks very exciting for tutors and students alike. I can’t wait.

Mike Pennington

Mike is Chairman of First Intuition Ltd, PQ Magazine’s College of the Year in 2010, 2012, 2014, and 2016.



A number of leading examination and professional bodies enjoy a successful working partnership with BTL. Below are a number of case studies that show how we work with our partners to develop and deliver their successful, global e-Assessment programmes using our Surpass Assessment Platform.

Featured Case Study

City & Guilds

Featured – City & Guilds

City & Guilds have been working together with BTL since 2010 . In this time, they have delivered almost 10 million on-screen tests (at the time of writing) around the world, including some extremely secure and difficult locations.
In these short interviews, Granville Edwards explains in his own words how “Evolve”, a City & Guilds platform built on Surpass, assists them in achieving their goals, the challenges they face, and what it means to be a member of the Surpass Community.

“People trust in the City and Guilds brand. A lot of people see City and Guilds as the stamp of approval for a level of training. We’re obviously very keen to make sure that all the products we work with, all the people who work with it, and what we deliver to our centres has got that level of quality with it.” Granville Edwards (City & Guilds)

Government regulated school leaving and university entrance


“I was given the role of developing two new A-Levels, and, on my arrival what I actually wanted to do was make those A-levels paperless.” – Allan Perry (WJEC)
WJEC are a pioneering awarding organisation based in Cardiff. Following a very thorough development process, in 2009 they delivered the very first on-screen only general qualifications to England and Wales with the help of the Surpass Assessment Platform.
In this 4 minute case-study video, Allan Perry (Subject Officer) and Ian Morgan (Acting Director of IT) explain why it was important that these new examinations were developed with a new generation of candidates in mind.

“One of the things we’re trying to do is get additional value out of the BTL Software [Surpass] so we can use it for on-screen assessment… but we can also then use the processes and the workflows which are inbuilt to the BTL system, so we can support our paper based assessment mechanisms as well.” Ian Morgan (WJEC)

Starting with Allan’s arrival at WJEC, the introduction explains how his vision was achieved by working together with BTL and the Surpass Platform.
Ian and Allan then go on to discuss how taking advantage of the extra functionality and processes available in Surpass has positively affected the organisation’s collaborative workflow and helps them adhere to strict security processes when working with traditional paper-based examinations.
Finally, their journey is complete when Surpass is used to mark exam scripts electronically; efficiently distributing detailed scores and allowing teachers to make innovative use of results in candidate review sessions.

Case Study highlights include:

  • The challenges and rewards of using technology in the general qualification process
  • Using technology to improve general qualification assessments
  • The benefits of using an on-screen marking system, including teacher and student benefits
  • Using an on-screen assessment platform to improve the paper based process
  • The benefits of having one digital system for collaborative processes


“AQA has a really pivotal role as a charity in showcasing how an exam board can deliver not just assessment products but a really valuable and empowering assessment service.”

AQA is the largest of the unitary awarding bodies, delivering general qualifications (GCSE and A-Levels) in the UK. In this series of short video clips Dave Mellor, Judith Rowland-Jones and Seb Ross talk about the challenges they face in the modern assessment environment, and how experience combined with a practical use of technology helps their organisation to improve logistical processes, while also developing new systems to ensure every candidate gets the result they deserve.

“AQA has a really pivotal role as a charity in showcasing how an exam board can deliver not just assessment products but a really valuable and empowering assessment service. And that’s in terms of the design of the assessment tasks, in terms of the marking, in terms of the way the evidence is kept and held securely, in terms of the authentication processes to ensure that every student gets the result that they deserve…” – Seb Ross (AQA)

Download the AQA Case Study Playlist Transcript

This case study playlist covers several topics, including:
  • The challenges and rewards of using technology in the general qualification process
  • Addressing the challenges of delivering on-screen general qualifications
  • Using technology to improve general qualification assessments
  • Using SurpassPaper+ as a tool to evolve paper-based assessments
  • The use of technology in schools
  • The benefits of using an on-screen marking system

Looking for something more specific? We have a range of case studies and testimonials grouped into topic specific playlists on our YouTube channel.

SQA Biotechnology

Find out how BTL e-Assessment solutions helped the SQA deliver Intermediate 2 and Higher external assessments on-screen.

SQA Biotechnology Case Study

Functional and essential skills

Chris Jones, CEO of City & Guilds, talking at the BTL 30th Anniversary Forum

Chris Jones offered a strong case study from City & Guilds, telling the story of how they quickly embraced e-assessment in 2002 and have not looked back since. Technology has changed rapidly and is a game changer for education, being core to the modern success of City & Guilds. He addressed how E-assessment is now the primary testing channel for City & Guilds, with paper very much the exception.
City & Guilds assess millions of exams over the year, awarding over 2 million qualifications across 80 different markets.

Download the Transcript of Chris Jones, CEO of City & Guilds, talking at the BTL 30th Anniversary Forum

If this volume of tests were delivered by paper, it would not be logistically appropriate, and would result in a negative effect on the students. Chris shared his desire that paper assessment should be phased out with the main reason being simple: e-Assessment provides a better service and City & Guilds is a shining example of how this can work and the benefit to the business it can bring. Since City & Guilds started moving to e-assessment, they have certificated 16 million online tests across the world. However, these changes have taken time, and must continue to be done in such a way to ensure that the quality of the experience is not compromised.
Partnering with BTL for the last 5 years has worked well for City & Guilds, due to our understanding of education, training, and the role of assessment and technology. Chris Jones stated that City & Guilds are proud to work with BTL and look forward to a continuing partnership.
Technology has plenty to offer the development of assessment and by working with companies like BTL, City & Guilds believe that technology is an opportunity and not a threat, or a risk, for education institutions and work alike. The benefits of e-assessment are clear, benefits simply not possible with pen and pencil. To make the most of this exciting future, changes need to be made to the availability of broadband and technology in within the classroom; it’s too important not to invest in this, and we all have a role to play in embracing technology to improve the delivery and measurement of teaching and learning. BTL are playing an important role with working with City & Guilds to bring about change.

The City & Guilds Functional Skills ICT Simulations Test Project

For nearly a decade, City & Guilds has offered e-Assessment on the Global On Line Assessment (GOLA) platform. From a summative assessment perspective, this has been solely a multiple choice offering. City & Guilds has, however, become increasingly aware of the need to introduce more contemporary and sophisticated assessments in line with current pedagogical thinking and technological advances.

Learn how City & Guilds and BTL worked together to create test ICT simulations for their Functional Skills programme.

The City & Guilds Functional Skills ICT Simulations Test Project


The Royal College of Paediatrics and Child Health (RCPCH)

Learn more about how RCPCH are providing a better assessment for their students in this video:

“What would I say to other Royal Colleges…? It (Surpass) is going to provide a better exam for your candidates. It’s going to be more ‘Real World’. We’re improving something that’s already recognised, globally, as being a world quality exam, and the outcome from that is a better outcome for children.” – Dr. Will Carroll (RCPCH)

Download the RCPCH Case Study Transcript

The Royal College of Paediatrics and Child Health (RCPCH) Presentation

Case Study Focus: First steps into computer based testing In this User Group presentation, Julia O’Sullivan from RCPCH talks about how the Royal College took their first steps towards computer based testing, and how working with BTL to implement Surpass has helped them stay on track and address problems and concerns with making the leap from paper to onscreen.
This presentation was filmed before the full implementation of Surpass (which has now happened) and as such, Julia O’Sullivan talks about how a pilot scheme was initiated, and how the valuable data extracted from the scheme was used to shape the final delivery of the platform.

“One of the things BTL do, which none of the other companies were doing, was they invited us to take part in a workshop before the final presentation. So when we got going to our last three companies, they invited us to a workshop to go through everything we wanted to do, and work through some of those processes, so they were starting to gain real understanding around things, and the final proposal they presented showed that they did understand what we were about as well.”
“…They sourced the test centre network …we said where we deliver the exam, which is all over the UK, and overseas, and they sourced all these test centres for us, so I think they worked really hard to do that.” – Julia O’Sullivan, RCPCH (To find out more about how we’ve since expanded our test centre network, call us on 01274 203 250 )

Download the RCPCH Presentation Transcript

Higher education

In this series of interviews, staff at Windesheim University of Applied Sciences in the Netherlands explain how Surpass: The Assessment Platform touches on each of their unique roles, and the benefits this brings to the faculty and their students.

Windesheim University of Applied Sciences

Introducing Surpass to the Faculty and Maintaining Exam Quality

In these short interviews, Mark and Hans explain how, as business analyst and technology advisors, they worked together with faculty staff to introduce a new Software Platform, and how the university has used the opportunity to ensure a high level of quality is being maintained throughout the question authoring process.

“Then they realise, ‘Hmm, I can create a test using randomising rules, which creates a new test but I don’t have to, every three or four months, create a unique test myself again; I just click on the button, and there it is!’” – Hans Selles

“Additionally, what they also prefer is that they are able to test in a different way. For instance, our sports and education studies can test with video and that’s a nice way for students as well to be assessed in specific fields.” – Mark Heupink

Usability – Using Surpass to Easily Create Better Assessments
As both a lecturer and Chair of the Examination Board, Kirsten has a lot of experience with assessment technology. Talking to us in her native Dutch, she explains the advantages of using Surpass over other software and how its simple usability helps her workflow.

“With the previous software, I had lots of problems adding tables and graphs – which are useful for a subject such as economics – to a question, and that is just so much easier in Surpass.” (translated from Dutch) – Kirsten Huising

The Advantages of Digital Testing for Educational Staff
Louis is an IT manager at Windesheim and responsible for the implementation of Surpass. In this short interview, Louis talks about how large volumes of tests are delivered concurrently across the campus, and how Surpass helps to ensure a reliable, simple-to-use workflow.

“To me, the biggest advantage in digital testing is that once you’ve set up your test, you’re ready to go multiple times. There’s no work left over for the second, third or 44th test!” – Louis Klump

Windesheim University (Hans Selles Presentations)

In these two presentation videos, Hans Selles from Windesheim University of Applied Sciences in the Netherlands talks about various aspects of digital testing and how Surpass assists in managing these processes.

Saxion University of Applied Sciences

“We think we are heading towards more formative tests in Surpass, because they help to predict where students need to study.”

With thousands of students from countries across the world, Saxion University of Applied Sciences in the Netherlands is a pioneer not only in its own country, but within the Higher Education sector in Europe.
A long-time user of on-screen assessments, Saxion started using Surpass in 2012 when they were looking for a system that could better meet the current and future needs of a large, modern, multi-campus international university. Chosen following a rigorous selection procedure, BTL and Saxion have worked together to ensure a smooth transition and enjoy working together as partners, creating a better future for assessments.

“We use Surpass to guarantee quality…producing the right tests and the right test matrix…” – Leen Van Kaam (Saxion)
“The migration to Surpass was a big effort, but everyone sees what we can do now with Surpass, that we couldn’t with the application before. The relationship with BTL is very nice. You can say what you want… and within the year you get what you want in the application.” – Alwin Wullink (Saxion)

In this short video, Leen and Alwin from Saxion University provide a fantastic insight into the way Higher Education establishments in the Netherlands handle their assessment processes and work together to create a better future for their education system.
Starting off by talking about how Surpass allows the hundreds of potential test authors to create a better quality test by working together, the topic moves onto Saxion’s relationship with BTL and the migration from their old assessment platform to Surpass, which is now used for authoring, scheduling and delivering tests, as well as reporting and analysis of test performance.
Finally, Alwin shares his thoughts on the benefit of digital testing throughout the university, while Leen looks towards the future of more formative testing using the system to further improve examinations.
Video interviews from other members of the Saxion faculty will be available in the future.

This case study playlist covers several topics, including:

  • Migration from one system to another
  • Using technology to improve the quality of assessments
  • Student benefits from digital testing
  • Impressions of the working relationship with BTL

Looking for something more specific? We have a range of case studies and testimonials grouped into topic specific playlists on our YouTube channel.

Saxion University (The Netherlands)

Case Study Focus: e-Assessment integration for large scale summative testing. In this case study presentation, Leen Van Kaam, a System Administrator from Saxion University in The Netherlands talks about the integration and scalability of the Surpass platform, and how the system was chosen following a strict tendering process. She also mentions that following the implementation of Surpass, all 12 of the University’s schools have been successfully transferred into one solution, and are currently delivering between 40 – 50,000 summative assessments per year.

“We had 12 different schools, almost independent, and they all were used to doing their own thing, so they had to have their own centre we they were able to do their own thing with their own roles and privileges. Privileges are very important, for instance a lecturer in Saxion never develops an exam, he only develops questions and he’s making on paper an exam matrix, and that’s his work.” – Leen Van Kaam, Saxion University

Download the Saxion University Presentation Transcript


EAL e-Assessment Case Study

EAL (EMTA Awards Limited) is the UK’s leading awarding organisation for engineering National Vocational Qualifications (NVQs) and their Scottish equivalent, SVQs. EAL offers e-Assessment for our VRQ qualifications and Key Skills.

This case study provides an insight into how EAL delivers their e-Assessment programme using the Surpass® e-Assessment Platform.

EAL: e-Assessment Case Study

AAT Computer Based Pilot

Find out how the AAT and BTL successfully undertook their first on-screen delivery of their live exams.

AAT Pilot Case Study

Test Delivery Services

Pitman Training (Test Centre Network)

Pitman Training deliver a large number of assessments for BTL’s partners and customers. In this short interview, Mahar Uddin explains how a test centre easily prepares for delivering on-screen assessments using Surpass, the number of exams delivered, and some of the challenges involved such as internet connectivity issues and uncooperative candidates.

“Surpass makes it very easy to deliver exams…It’s a testament to the platform, how robust it is, that we’ve never had to call the help desk; but we know that the help desk is always there.
[It’s important to have a rigorous preparation system because] Any time you invigilate, you’re invigilating to protect the exam.” – Mahar Uddin (Pitman Training)

Reflections on assessment

Surpass community experiences


In this presentation (recorded at the 2016 Surpass Conference in Edinburgh) long-time Surpass user Becky Vergnano documented how NCFE handled the original implementation of Surpass, and the lessons learned along the way.
The talk is full of great advice that can be shared with other organisations just starting out in the field of on-screen testing and assessment software, and is also a good refresher for all e-assessment users in thinking about how they can make the most of their assessment platform.

Surpass community experiences playlist

Testing on tablets

Paper-based testing with Surpass

On-screen marking in Surpass

Using technology in assessment – BTL Westminster Forum event

Download the Transcript of Chris Jones, CEO of City & Guilds, talking at the BTL 30th Anniversary Forum
Download the Transcript of Prof. Peter Twining talking at the BTL 30th Anniversary Forum

Using a Test Centre Network

Getting Started With e-Assessment
At the 2016 Surpass Conference, several new users were happy to share their unique experience of embracing e-Assessment.
From each individual account, you can see how different organisations have handled the initial setup and integration, their intentions for rolling Surpass usage out further, and the impact they expect it to have on their business.

Experience first-hand how and why CILEx selected Surpass in “Customer Experiences” with Vicky Purtill, from the Chartered Institute of Legal Executives (CILEx).

“Why did we choose BTL? Recommendation of others was really important, they had experience in the Awarding Organisation sector, which when you’re OFQUAL-regulated (as well as Qualification Wales and CCEA) and delivering regulated tests, it’s really important that you get it right.”

Discover how Surpass has helped to overcome the challenge of global test security and remote delivery in “Customer Experiences – The Welding Institute (TWI)” with Maurice Whittaker.

“We had people recording exam papers on their pens and putting it onto YouTube. We’ve had people broadcasting the exam via the same mechanism to people outside the room who are sending the answers back…”

Hear Menno de Vos from NHTV Breda University (the Netherlands) talk about their previous experience with another supplier, and why Surpass has been chosen to re-ignite the e-Assessment process.

“2020 is the goal to have all of our academies working together in Surpass… If I talk to educational managers, they all want it now…”

Professional accreditation

Association of Accounting Technicians (AAT)

Case Study Focus: Marking high stakes assessments This presentation was a joint delivery by BTL staff and Marie Larusson of AAT addressed the different ways powerful tools like Surpass can be used to facilitate and improve marking of high-stakes transcripts, taking advantage of the intuitive user roles set within the software.

“We also separately offered some paper-based exams, but only to students that qualified for reasonable adjustments or special considerations. For any other students, we stopped offering paper-based in December 2013 when we had our last sitting. All the new qualifications that we launch from now on are only offered as computer-based on SecureAssess.” – Marie Larusson, AAT

Download the AAT Presentation Transcript


Case Study Focus: Introducing a new e-assessment platform to replace an existing system In this case study presentation, Clare Ruiz-Palma from Cache talks about the selection process and integration of an e-assessment solution, from the perspective of a client who wanted to change providers.
This particular study is a fantastic example of a customer being able to tell us not only how they’ve made use of existing Surpass features, but also how they are able to plan future improvements by working with us to feed into the Surpass product roadmap.

“We came down to three products, and in the end – after going through a similar sort of tick process – Surpass came out as the clear winner. The reasons that we went for us that it matched our requirements, particularly one of the requirements we needed was block-booking of candidates, and also it was fairly priced and also about the company. Some of the companies we looked at were a bit smaller, we weren’t totally sure how it was going to go with their own app.” – Clare Ruiz Palma, CACHE

Download the CACHE Presentation Transcript

0 thoughts on “Case Study Computer Based Examination System In Ancient

Leave a Reply

Your email address will not be published. Required fields are marked *