Over the past few months, MAP has taken an in-depth look at our practice of providing reviewer and panel feedback to applicants. We believe its greatest value is as a communication loop — after having gone through the vulnerable act of submitting their dreams, plans, budgets, etc., to an anonymous body of reviewers, through feedback, applicants have confirmation that their work has been seen, and their ideas have been heard. Honoring that communication loop, and by extension honoring the work of each and every artist behind the applications, regardless of the outcome of the selection process, is an essential value at MAP.
At the same time, we want to acknowledge a few unique challenges that feedback, as we have typically provided it, presents to MAP’s lean administrative staff and to share our questions about whether our methods are in fact delivering useful information.
Our examination has led to some changes for the 2018 grant cycle that we believe will improve the process, without eliminating that important channel of communication between program and applicant.
Round One Feedback
In the past we have provided feedback at the end of Round One for about 675 proposals that will not move on to panel. This entails staff members re-reading each proposal, aggregating and revising comments from approximately three thousand open comments fields (since each application is seen by 5 unique reviewers) and revising those comments for clarity, context and the removal of attribution. The problem with this method is twofold: open-ended comments fields means notes, while sometimes enlightening, may or may not be relevant to the artist’s own project goals, and may or may not help the applicant improve future proposals. In addition, the amount of human resources required to deliver so much feedback is not sustainable for our small staff. We think the following three changes will improve the situation:
- Applicants will have access to the precise guidelines and scoring rubric we ask reviewers to use in their facilitated assessment of proposals. Fully understanding the reviewer perspective will help applicants create strong narratives. Read those materials here.
- The primary focus of MAP staff’s support of applicants will be on providing systematic support prior to the Round One deadline. This will come in the form of applicant-directed feedback on grant narratives at a wider and more organized scale than we have provided in the past. More information will be live on the website on August 28, 2017.
- Every applicant that does not move forward to the panel phase will automatically receive the reviewers’ scores.
Round Two (panel round) Feedback
We have typically provided feedback on proposals that moved to panel using the same method described for Round One, while also adding panel notes, again revised for clarity and to remove attribution. We see two problems with this method of feedback.
First, the relatively small number of applications that move to panel (approximately 8 percent of the total pool) have already been read and positively scored by five or more reviewers. Applications at the panel stage are by definition strong and well aligned with the program. When such applications are not selected as grantees, it’s more often than not due to variables that have little to do with the application itself, such as the unique demographic, geographic, and aesthetic composition of the overall pool the panel is looking at. Therefore, panel notes on a specific application can not be fully appreciated or comprehended outside the context of the full, wide-ranging 3-day panel conversation.
The second problem is that, by design, the panel is a rotating body, made up of a different group of people each year. This practice allows many different perspectives and communities to be represented at panel over time. While MAP firmly moderates the panel conversation so that it adheres to the program’s criteria, nonetheless because of the unique makeup of individuals each year, the locus of a given panel’s concerns are impossible to predict. Therefore, notes that an artist receives in 2017, for example, may not necessarily apply to the concerns of a panel in 2018. (The things that will improve an application in a meaningful way from year to year are the grant-writing skills that MAP will support prior to the first deadline).
To address these issues, MAP will make the following changes to Round Two feedback in 2018:
- Applications that move forward to the final review round will receive a second round of writer-directed feedback prior to submitting their Round Two materials. More information about that process will be available later in the fall.
- MAP will invite a non-voting panel observer to write a narrative overview of the proceedings to be shared with all applicants at the conclusion of panel. Application-specific notes will not be provided post panel. For only those proposals that went to panel but were not selected for a grant, a follow-up meeting with MAP staff will be available at the artists’ request.