Intermediate Cultural Impacts
Understanding perception and satisfaction with the event and host city takes into account :
- Visitor perception (general) and satisfaction (individual event aspects and overall event)
- Local resident perception (general) and satisfaction (individual event aspects and overall event)
- Civic pride
- Local valuation of an event
Intermediate impacts around identity, image and place focus on understanding the perceptions of residents and visitors about the place which is hosting an event.
Measuring Satisfaction
Understanding satisfaction is an important operational tool for host cities as the more robust evaluation of satisfaction with the event will lead to a more accurate understanding of participants’ needs.
The key will be in understanding the quality of participants’ experience through their perception of different elements of the light festival or event. The focus should thus be on measuring those perceptions and analysing the resulting information.
Participants’ perceptions (be it local residents or visitors coming from elsewhere), about the event are best captured through survey work. The basics of surveying have been covered in the Survey Development section.
Additional points specific to measuring satisfaction include:
- Pre-event experiences are best measured at the event using self-completion surveys
- Some elements of satisfaction cannot be assessed without first having experienced the whole event which needs to be considered when designing and timing the survey and considering approaches to be taken.
- If specific target groups have been identified (e.g. overseas visitors or local residents from particular communities), more detailed feedback using qualitative techniques (such as focus groups) might be beneficial in supplementing any surveys.
- Follow-up surveys after the event had taken place can be used to derive more informed opinion on key elements, or where additional detail is required. For example, in Osaka the research group sent out online questionnaires to the people registered with a research company and who also visited Osaka Hikari-Renaissance.
When evaluating satisfaction, the simplest way is to develop a five-point scale which attributes scores both positive and negative aspects of the event (from very satisfied to very dissatisfied), to allow for net satisfaction rating for these selected aspects. Typical areas to measure include event format, quality of publicity and quality of service.
Host cities need to consider satisfaction scores alongside the number of respondents and responder characteristics, in particular if the event is designed to attract particular sub-groups or type of visitors/participants.
If each aspect of satisfaction assessed is of equal importance, then the overall satisfaction score can be delivered based on an average of all the scores recorded across the range of aspects. Alternatively, host cities may put particular emphasis on some elements than others and in this case it may be appropriate to weight responses to reflect the relative importance of each aspect.
Additional sources of feedback from participants include blogs or website comments from their own event or another similar event.
Measuring Perceptions
The main vehicle for measuring the impact of an event on people’s perception of identity, image and place will be primary research at an event, or post-event using for example telephone, electronic or postal surveys.
The tool to measure perception is very similar to the one to capture satisfaction. The minor difference between the approach employed to capture satisfaction and the approach employed here is with the set of standard responses provided.
For satisfaction, people were asked to respond on a scale from “Very Dissatisfied” through to “Very Satisfied”. To measure perceptions of place, people were asked to rate their agreement with a series of statements on a scale from “Strongly Disagree” through to “Strongly Agree”.
Civic Pride
To assess civic pride resulting from hosting major events, questions can be asked of attendees and non-attendees using simple closed questions. Sample questions which might be included in a survey are:
- Does the staging of [name of the event] give you pride in your area
- Does the staging of [name of the event] make you think it is exciting or a more desirable place to live?
- Did the event make a positive contribution to the image, appearance or reputation of the local area?
As noted above, assessing the impact of an event on local people may well require measuring the opinions of those who have not attended the event.
Whilst a household survey could be considered, there are creative alternatives that may be more cost-effective or impactful. These include the inclusion of relevant questions on citizens panel surveys organized by local authorities, for example.
Local Valuation
Some events have attempted to place a value on the staging of the event in the local area. Contingent valuation techniques measure an individual’s “willingness to pay” for an event to be staged in the defined local area.
This approach attempts to quantify perceptions around the value of an event to an area, and provides a monetary measure which can be considered by regional authorities.
For example, Osaka attempted to assess the success of its event by testing audiences willingness to pay for the event. Osaka visitor survey included questions asking visitors for their maximum willingness to pay for the event. The value estimated amounts to 3EUR (medium) or 5.20 EUR (average).
Particpation in Cultural Activity
- Intent to participate
- Market penetration
- Market development
Light festivals and events can be particularly successful in making an impact on people’s intent to participate more frequently in a cultural activity. Whether this translates into action may well be dependent on the effectiveness of other organisations and the availability of development programmes to support the event. Nevertheless, the potential for an event to inspire people is an important one and should be measured.
Establishing intent to participate can be captured using the same basic surveying and assessment approach detailed above. In summary, respondents are asked to rate their agreement on pre-defined scale with a number of statements which link their engagement with the event with an intent to change their participation in an activity.
When assessing intent to participate, it can be useful to group respondents by their current participation levels. Using the baseline data gathered on target audiences, it is possible to disaggregate “market penetration” effects from “market development” effects. In other words, does an event simply act as a catalyst for increasing engagement by a predisposed population, or does it genuinely enthuse new audiences?
Examples of assessing participation:
- Museums at Night is the annual after hours celebration that sees hundreds of UK museums, galleries, libraries, archives and heritage sites opening their doors for special evening events during one weekend in May. In 2010, close to 300 organisations hosted 85,000 visitors across three nights of which close to half (47%) were new to the museum they were visiting, and 97% of visitors stated they are inspired to visit other museums. Evidence from evaluation suggests that the event encouraged lapsed attendees to come back into a museum (20% of visitors had not been to a museum for more than a year) and reached new people (5% of visitors had never been to a museum before).
- After Hours – Science Uncovered event took place at the National History Museum in London on September 2010 as part of Researchers? Night, a Europe-wide festival of research. The key objective of the evaluation was to assess the impact of the event on public perception of researchers and their work. The great majority of event participants stated they had benefited from the event and learned something new about science or scientists.
