#EconTEAching chat: 5 examples of assessments

On March 29th we had the first #EconTEAching chat (joint collaboration between CTaLE at UCL and Warwick). It was a well-attended, friendly conversation on my previous entry on how to change assessments in a way that we can offer a good response to the current crisis.

While we talked about the type of assessments, we may also need some inspiration and examples to start with. Here I talk about some of assessments I have adopted in this period (but not only). I don't mean these examples are exhaustive of what can be done, or even arguing they are the best we can do. You can simply use these as inspiration and adapt these to your courses. Remember that assessments are usually set in line with university's regulations (and limitations), which means I have adapted my assessments to fit these.

Given that I mostly teach Macroeconomics, most of the assessments are in this field.

1. 24h Exam: Monetary Policy/Money & Banking

For moving from invigilated exams to remote assessment, I opted for 24h exams rather than time-limited ones (e.g. 3h). I discussed the benefits of these here, just to mention few, it avoids issues with late submissions or network accessibility: once the student is able to download the instructions, it doesn't rely on access to internet to write their answers, and will need access only for submission, with a long time-frame. Also, given that they may need graphs/equations in their answers, there is more time for them to create these and added in their work.  

For Money and Banking, students will get four questions, two which required short answers (100 words max), two which require longer ones (500 words max). The following are examples from past year exams, but with some adjustments, I think these could be also used in a 24h exam (for the longer answers): 

Question 1: 

In the FT article ‘Bank of England allays fears over leveraged loans growth’ (23 Jan

2019) we can read:

“Regulators across the world have flagged leveraged loans as an area of concern, pointing to declining lending standards and falling protections for investors as a possible indicator of financial risk [...]. ”

Use a balance sheet to explain why regulators may be concerned about the increase in leveraged loans in the UK economy? What difference would it make to the strength of UK monetary policy if all mortgages were fixed-interest rate loans (in the short run and in the long run).

Question 2: 

In February 2018, Mark Carney, governor of the Bank of England (BoE), said the Monetary Policy Committee (MPC) “expect(s) that in order to return inflation sustainably to target ... it will probably be necessary to raise interest rates ... somewhat earlier and to a somewhat greater extent than we had thought in November”. The BoE governor also said in a letter to the UK chancellor that until now the MPC had been willing to tolerate above-target inflation for a longer period “in order to support jobs and activity at a time when uncertainty was elevated and the economy was slowing”.

(a) Using the 3-equation model, discuss the relationship between inflation rate and economic output; and how this may help explain Mark Carney’s statement.

(b) Explain the transmission channels of how interest rate will affect inflation and aggregate demand in the UK economy. Can you think of some factors that may affect how well these transmission channels work (you may want to use some ‘real world’ examples in your explanation)? 

For changing these to open-book assessments, I would emphasise more the policy aspect of these two questions. In question 1, I may add something about the difference in mortgages and how monetary policy affects distribution among lenders and borrowers, and within borrowers. In question 2, I would be more specific about the discussion on transmission channels. For instance, I would ask the effects of very low interest rates on market expectations and more in general on financial investors behaviour, perhaps asking to discuss the unexpected risks of low interest rates that are not considered in the transmission channels. 

For these questions, having access to the resources (books, slides) of course is an advantage, but students would not be able to fully answer these if they haven't studied in advanced. Merely repetition of the textbook or readings material (even if using their own words), would allow them to get the basics and get a pass mark, but not much more. 

Students of course are also able to communicate with each other. Communication will help them to further understand the questions and exchange ideas on potential answers, but given that there is no a one correct answer, I expect an improvement in the median answer, which will be taken into account while marking. 

For each question, students also have a word-limit, which means they have to be selective in the information they use to answer the question and they can't just write everything they find around. Those who have studied will find easier to assemble a good answer, those who haven't will likely be more tempted to either based their answers on the basics of the book (e.g. the 3-equation model in this case) or what they find online, but likely it will lack a good structure. 

Of course, all answers will go through a plagiarism software (Turnitin).

2. 48h Take-home assessment: Inequality

 This is for Applied Macroeconomics (Year 3) in which I delivered half of the course. Similar to 24h exams, but arguably, this will require even more application of knowledge from the students. This year we talked about 'Inequality'. In order for students to get familiar with the new structure, I made a mock exam available and had a synchronous Q&A session. There is also a forum online for questions (for the subject or the new exam structure). 

Have a look at the mock exam here.

As before, students benefit from accessing the resources (mostly academic papers) but more than in the previous case, they really need to know what to do with these resources, therefore they should have studied for this. Arguably, this type of assessment simulates the 'real world' more than the typical invigilated exam. Those students working in policy-related jobs need to produce policy briefs, but more in general economic briefs are widely used. 

3. Online Multiple Choice

We use few multiple choice test for low-stake assessments in large classrooms. These are easy to mark, ideal for very large groups. However, good multiple choice assessments are time-consuming to set/create. 

This year, we had to move the test from an invigilated one, to online. Students had a one-hour to provide their answers from the moment they accessed it, but they had a 3-hour time window (9am - 12pm BST) to access it, e.g. if students access the test at 10am, they had to submit their answers by 11am. I was really impressed on how the whole system worked and I will try to keep this format for the future, even if we return to face-to-face delivery. 

Regarding poor academic practice, while some students may have colluded, the distribution of marks was not so dissimilar to the invigilated test they sat in Term 1. In order to minimise collusion we adopted few measures: 

• For each question, there were more versions of the same question, 
• Order of questions were randomised,
• Order of answers within the questions were randomised, 
• There could be more than one correct answer, students had to select all correct answers (I didn't use negative marking). 

Creating the various versions of the questions was a bit time-consuming, but the marking is done automatically, so this partially compensates for the time. Some questions require students to make calculations, some to interpret an article, very few required students to simply remember a definition or similar. So, even if students benefited from using books/lecture notes, this will affect the time available to answer other questions. 

I made a mock-test available beforehand, so students could familiarise themselves with the software used (around 60% of students used it). Students also had some online quizzes to practice. 

An example of one of the MCQ: 

Consider the economy represented in the graph below. This economy has been hit by a positive demand shock (represented by point B). Before the shock the economy was at equilibrium (represented by point A). The top graph represents the equilibrium in the goods market, the labour market is depicted in the middle graph, and the bottom graph shows the central bank's monetary rule.

<< Figure: Positive Demand Shock in the 3-equation model >> 

Given this information, which of the following statements are CORRECT:

(a) In the model, the change in interest rate by the central bank will make the Monetary Rule curve (MR) shift upwards to cross the Phillips Curve (PC) at point B in order to bring the economy back to equilibrium.

(b) In order to bring inflation back to target, unemployment will have to increase above the equilibrium level during the adjustment process led by the central bank.

(c) If the demand shock is permanent, once the adjustment process guided by the central bank finishes, the economy will be at a new equilibrium point with higher potential output and higher interest rate. 

(d) The responsiveness of investments to interest rates matter for central bank's

response to shocks.

There were other versions of this question that were very similar (to guarantee fairness among the questions). Variations of the question can consider a negative rather than positive demand shock, some variation in the answers (e.g. which curves shift), the type of shock, etc. 

These tests are usually low-stake and I wouldn't rely much of the final mark on just multiple choices, but are useful to keep students engaged with the subject. 

4. Group Report: An opportunity to assess more than one subject? 

Sometimes we underestimate the value of group work. Collaboration in an important professional skill highly valued by employers as the majority of jobs require to work in teams. Moreover, group work creates more opportunities for critical thinking as students have to share their ideas with each other which promotes student learning and achievement. Another advantage of group work is that marking is significantly reduced as students usually submit one work per group. I also believe it's less likely that students engage in poor academic practice such as using essay mills to produce their work. 

This assessment was not in response to the current situation, but I believe it can be adopted and adapted in case we needed it. The idea is to provide students with a scenario in which they have to provide a response as economists. During the course, we analyse how macroeconomic variables such as inflation, unemployment, output interact in the short-run and how shocks to the economy affect these variables and the assessment was based on this. 

Group work: 

You just got hired as part of a team of economic analysts for a car manufacturing company in the UK that sells cars all over the world. Your boss is a graduate in Engineering with an excellent knowledge of the car manufacturing sector, and she needs to make some business decisions for 2020. 

Your team has been given the task to produce a macroeconomic analysis of the current UK (macro)economic environment and how this may affect the manufacturing sector. For the calendar year 2019, the company’s retail sales were down 5.9% compared to 2018. This report will be used for company’s future investments and hiring decisions in the UK market. 

You can see the instructions I made available for the group report here.

I emphasise that this is a macroeconomic report, but they could easily write something similar for microeconomics, for instance, looking at the structure of the market in which the company operates. In fact, more integration between modules can enhance students' engagement and potentially promote higher order learning. 

5. Group Video project: Communicating with the general audience

In the Macroeconomics module I teach at Warwick (with Natalie Chen), we have been using this assessment for couple of years now, so it is not a response to the current scenario, but I believe it can easily been adapted and we will continue to use it even if we go online next academic year. This is strongly inspired by Dimitra Petropoulou's work, with some changes.

The advantages of this assessment are the ones that Dimitra et al. mentioned (scalability, employability, increase in student engagement). Even if we don't use peer-assessment for marking and the videos are marked by the teaching team, it still saves a lot of time marking if compared to typical essays (that we previously used). Other differences include that we provide the topic for the video, and the marking criteria used. This last point makes the video easier to mark. The video is marked out of 5, we provide five marking criteria. If the video meets the criteria, they get the mark, if the video doesn't meet the criteria we provide a comment why this doesn't.

You can see the instructions for the 2019-20 project here, including the marking criteria.

Few considerations for setting group-work when working remotely:

• Students may not have the opportunity to meet each other in person. It may be good to set up a forum within the VLE to allow students to exchange ideas/talk. This would be good practice in any case, but more under the current scenario. 
• Explain students the extra transferable skills they acquire by working on this assessment. The capacity to work remotely will only be more valuable in the near future. 
• In the instructions, make students aware of time-zones and the flexibility require when setting meetings. 
• Provide some information and guidelines on how to work remotely. It may seem obvious given the level of technology available, but providing a basic starting point may help, if only to set everybody on the same page. Remember that different technologies dominate different countries, and while for us it may be normal to communicate by WhatsApp, in China WeChat is  preferred. Students may be better at using technology, but they need to be aware of these differences.  

It is important to share what we are doing in a more practical way. This is my attempt, and if you feel inspired please share your interventions too! If we put all our ideas together, we can help colleagues that are having a rougher time, but also we can improve ourselves.