Recently I heard someone who had taken one of the IT certification exams complaining about questions that test memorizing the facts. He was also not happy with the "twisted" language that he had found in some of the questions.
Let me give my perspective - from the educational/cognitive standpoint - provided I have been developing content and questions for a long time as well as taking several exams, especially in the past when I was still an engineer.
The first complaint was: "There is too many `memorizing` questions."
No matter that testing the facts (e.g. commands and parameters in IT business) is discouraged in exams (the theory is in favour of "application level" comparing to "recall level", based on the Bloom`s taxonomy), memorizing some commands and parameters is mandatory, especially for top certifications such as Cisco CCIE. It is true the facts can typically be searched in documentation or googled. But, top engineers, such as CCIEs, are often "deployed" when there is a serious problem in a mission-critical environment. In such cases there is no time to consult the manual. Imagine the pilots in their plane rumbling down the runway and just about to take-off. They do not have time to check the flight manual stacked in their flight bag at all. They have to know the procedure and what to do by heart. So, even in today`s world where everything is on-line and accessible instantly there are skills and knowledge pieces that have to be built-in into your brain. Just imagine you are troubleshooting a box that provides core connectivity and you do not have much backup. In such a case knowing the key commands by heart can save you a lot of problems and help you restore the network environment much faster. In such mission-critical environments time is that counts. So, memorizing some key commands and parameters is definitely a benefit, even a must, for a skilled engineer.
The second complaint: "There are unclear questions, sometimes the wording is a bit twisted."
I agree with this. The exam testing should check technical skills of an engineer and not their language skills. Even more. Taking into consideration the fact that many test takers are not native English speakers, they may even be in a de-privileged position. So from the standpoint of good exam writing the exam authors should pay their attention to how they structure the questions (stems). I checked one of the exam guidelines and found: " … stem is detailed enough for a successful answer." I read further: " … determine how a learner mastered a specific skill set, not how well the learner understood the question." Simply put. Avoid confusion or ambiguity due to a poor language in exam items. Returning back to my flight bag example: The manuals have to enable the skilled pilots to quickly find a solution if something goes wrong during the flight. So the content in manuals has to be clear and concise. The same applies to exam items.‹ back