‘I’m proud of our reputation in leading on professional examinations’ – John Hines  

John Hines is a Consultant Urological Surgeon, based at University College London, where he also holds an honorary associate professorship. He has recently completed his term as Chair of the Intercollegiate Committee for Basic Surgical Examinations (ICBSE). I sat down with the newly appointed Dean of Examinations to discuss his route to this role, his thoughts on the evolution of surgical examinations over the years, and his view on the future of surgical examinations within the Royal College of Surgeons of Edinburgh.

Could you tell me about your route into medicine – and surgery in particular – and why you pursued this career?

“I wanted to be a doctor from quite an early age, although there aren’t any other doctors in my family. I grew up in Southend in Essex, and went to Liverpool Medical School from 1980 to 1985, so I’ve been a doctor now for more than 40 years. It was fairly obvious that I was surgically minded from early on in medical school. I enjoyed urology during my student placement, and it happened to be the first job I ever did as a doctor. I was fortunate that the urology unit at the Royal Liverpool Hospital was fantastic, which really made my decision to pursue urology an easy one. I found it to be a very broad specialty and I had a very friendly team in the unit where I worked.” 

When did you first get involved in RCSEd and what drew you to become involved in examinations?

“My first involvement was, of course, passing my own surgical exams. In those days, these were the ‘old Fellowship’ exams, which were taken after we had been qualified for four to five years. There were some stipulations – we had to do a certain number of jobs and jobs of a certain duration. At this time, the four Colleges were all completely separate and the FRCS, which each College granted separately, was of surgery in general. There was no affiliation to any specialty.

“It was known at the time that Edinburgh was a great College and very friendly – I felt that we were helped and I felt that we were understood as candidates in a high-stakes exam. I also took and passed my specialty exam in urology in Edinburgh, which also drew me to
the College.

“I’ve always been interested in exams and education. At Whipps Cross Hospital, I was the Royal College of Surgeons’ Surgical Tutor for eight years and was the Associate Director for Medical Education, where I handled the surgical component of education. 

“After five years of being a consultant, I became an examiner for the Royal College. At that time, the College moved from the ‘old Fellowship’ examination to the standardised intercollegiate membership examinations (MRCS). I also became the Training Programme Director for Urology in my deanery and the Chair for Core Surgical Training in London. It was through these activities that I was invited to join the Internal Quality Assurance (IQA) committee of ICBSE in 2010. I then became a Supervising Examiner and then a Chair of the IQA committee in 2017, followed by Chair of ICBSE in 2020, just before the pandemic. Steering the MRCS examinations through the pandemic was very interesting and led to several changes, some of which have stayed in place ever since.” 

How have you seen assessment and evaluation in surgery change over the years?

“I’ve mentioned how things have radically changed even over the course of my own career, from the characteristics of the ‘old Fellowship’ examination, conducted separately by the four Colleges, to the standardisation and intercollegiate nature of the membership examination, and a separate specialty-focused fellowship examination later on in training. The next time the examination shifted markedly was during the pandemic, which was obviously a tremendous challenge for healthcare globally. From an examinations point of view, we felt we had to cancel one diet of the examination early on and, following this, we held a meeting with the ICBSE committee every week for 44 consecutive weeks to steer the exam through this period. We decided to shift our Part A examination online. This enabled a global reach for the examination, which was difficult to obtain previously through a paper-based multiple-choice exam, and we have stuck with this ever since. We also increased the amount of time between stations (from one to two minutes), which the Colleges felt made the exam much smoother to run. This rather small change was really well received by both candidates and facilitators, so we’ve maintained it since the pandemic. There is now another intercollegiate MRCS review currently ongoing, and I am leading RCSEd’s involvement in this.” 

How has assessment improved?

“I think one of the biggest improvements in my time has been the change from the separate examinations administered by each of the Colleges to the intercollegiate examination standardised with all four Colleges working together. This has enabled there to be much more communication between the Colleges – with regular meetings and a cross-fertilisation of ideas – which has helped drive innovation and move assessment forward. As a result, last year we examined 17,000 candidates across Part A and Part B around the world. 

“I also think the move towards a more standardised in-training assessment, with the Annual Review of Competence Progression (ARCP), is a much fairer way of assessing surgeons in training than what came before. As a result of these processes, we also understand much more about the standard and quality of the trainee reaching completion of training.”

How could assessment be improved? Could you tell us about some of the forthcoming changes to surgical examinations being proposed in the forthcoming review?

“One area that has been discussed during our current review of basic surgical examinations – and which is already in the process of being addressed – is recognising the differential attainment of candidates and ensuring the MRCS examination takes this into account appropriately.

“We also recognise that an important part of assessing a surgical trainee is the evaluation of their ability to perform operations. However, this may be best assessed without the framework of a formal high-stakes examination. We feel there should be a closer alignment between the summative MRCS examination and the in-training formative assessment of ARCP. We recognise that the consultants examining the MRCS and reviewing ARCPs are often the same individuals, and so having these systems unlinked is perhaps counterintuitive. How we should bring these two processes into closer alignment is being discussed at present.

“The influence of artificial intelligence, particularly in the form of question generation for Part A, is also part of this review. At the moment, one of the challenges with examining several thousand candidates across the world in a fair and robust way is having enough unseen exam questions that are relevant and varied. AI would be of particular use in generating these questions.” 

What do you see in the future for surgical examinations? Is there a role for surgical skill assessment?

“I’ve mentioned that there may be an influence of AI over the next few years, particularly in terms of question generation. Secondly, I do think there is a role for surgical skill assessment in the future and there may be a shift towards incorporating more simulation for assessment of surgical skills. We have simulation already, for example, regarding actors who are scripted to give the same standardised answers to questions in histories. However, there is certainly a role for procedural simulation within the examination and this, together with AI, may become the next stage of development for the MRCS over the next decade.

“I think it’s important for trainees to remember that this review won’t and should not result in drastic changes to the MRCS as it is currently designed. I think the MRCS remains a very fair and appropriate exam. The style of the questions may change slightly but, other than that, there won’t be any radical changes in the near future.”

Is there anything else you would like to tell our trainees about?

“I think we’ve talked a lot about the MRCS exam and briefly touched upon the specialty fellowship examinations, but I think it’s also important to highlight that the College has a great and longstanding reputation in conducting a range of examinations, including a number of professional dental examinations, perioperative care examinations and others. I am proud of our reputation in leading on such professional examinations and I’m very confident that we will be maintaining this reputation in the future. In this regard, we will also be appointing a Deputy Dean of Examinations in due course. In addition, we will soon be announcing the creation of a Guild of Examiners, which will reflect the importance of – and the pride we have in – our continued reputation of excellence in conducting professional examinations at this College.”