DG Cities has collaborated with Imperial College London for several years, from forming industry-academic partnerships for research projects to sharing real-world case studies for student learning. For the latest of these, we have been working with students on a key issue facing councils - the complaints process - and looking at how AI might help to streamline responses. For the next in his series on AI in local government, Graduate Consultant (and Imperial MEng alumnus), Nima Karshenas explains…

Capacity and deficit in a skilled workforce is often quoted as one of the main reasons for the shortfall in the ability of some councils to drive and implement innovation. DG Cities has been helping to address this by forging a unique collaboration with the EEE (Electrical and Electronic Engineering) course at Imperial College London, drawing on some of the very brightest minds in the country. This has been a resounding success over the years, and this year we tasked our students with reimagining the complaints processing system within councils, harnessing AI tools to ensure the robust, informative, and consistent collection and presentation of complaints data. 

We believe that every organisation providing social housing in the UK can benefit from this use case. The use of new technologies will improve the quality of service delivery, tenant satisfaction and reduce costs. Based on our analysis, this use case can be delivered with the current readiness of AI techniques.

Following two months of hard work, the students have provided us with an impressive proof-of-concept, leaving us with clear next steps to think about how to turn these systems into a reality. The potential is clear, and these are the beginnings of a long road towards making our public services smarter and more efficient; providing more value to the taxpayer, and most importantly freeing up the time and resources to allow a more proactive approach to governance. 

What were the outputs?

The students made use of a large language model (LLM) to process and categorise complaints, providing summaries, attributing them to their relevant department, and assigning them an urgency level to facilitate quicker resolutions. 

Dynamic Dashboard: The dashboard offers real-time analytics, enabling council members to identify trends and address issues proactively. The tile system allows for customisable insights based on the council’s priorities.

Interactive Map Interface: Complaints are displayed on an interactive map with markers that provide summary popups. This feature allows for easy visualisation of complaint locations and the status of each. 

Automated Data Handling: Complaints submitted via online forms are automatically processed and stored in a secure, online database. The integration of AI ensures that each complaint is categorised and summarised, reducing manual workload. 

Importantly, the data collected by the system can feed into more robust and detailed data analysis systems that can pool in other sources of data (IoT environment sensors, energy monitors, cameras etc.) enabling the council to develop evidence-informed response strategies to complaints, ensuring a prioritisation that matches internal policy and fairness goals. 

How does this system help?

Improved efficiency: The new system has the potential to significantly reduce the time required to handle complaints. By automating data entry and providing actionable insights through AI, the council can address issues more promptly. This shift from manual to automated processes helps eliminate backlogs and ensures that resident concerns are addressed in a timely manner.

Enhanced decision-making: The AI-powered insights and real-time analytics provided by the dashboard enable council members to make more informed decisions. Identifying patterns and trends early allows for proactive measures, potentially preventing issues from escalating and improving overall community satisfaction. The map-level UI enables the council to build a location-aware understanding of the issues faced by residents, allowing them to take appropriate engagement measures and problem resolution strategies. This ultimately means for more effective public services.

Greater resident satisfaction: With the ability to address complaints more efficiently, resident satisfaction is expected to improve. The system not only speeds up response times but also ensures that residents are kept informed through automated updates when their complaints are being addressed. This crucially brings the council closer to the community and ensures everyone can feel heard. Such a system has the potential to be extended to resident engagement in different contexts, such as digital inclusion.

Lower costs: Assuming an average of 50 daily complaints, the students have estimated the cost of using this model amounts to just around £4 per year. It’s important to note that even if this number is higher, model costs scale linearly.

Building organisational knowledge: Perhaps most crucially, developing  a system like this is hugely impactful to organisational knowledge and memory. Building out the data pipelines, codebases and organisational processes to maintain such a system will be crucial to massively accelerating the timelines of future AI projects within the council. Fundamentally, this is a resident engagement project, it has constructed an automatic means of collecting and sorting communications from residents. As such, it can be very easily adapted to other applications grounded in resident communications and engagement. Furthermore, building out these digital systems offers automated and robust collection of clean data (complete, correct and error-free), which will be crucial moving into the future. 

Next Steps

Address reliability of AI outputs: One of the primary barriers to making the system production-ready is ensuring the reliability of the AI-generated summaries and urgency levels. As these outputs directly impact how complaints are prioritised and addressed, they must be accurate and consistent. More extensive testing is required to validate the AI’s performance under real-world conditions. If performance is deemed insufficient, we must look towards a more sophisticated model, leading me onto my next point…

Fine-tune with council data:  The current model leverages general-purpose LLMs with Few-Shot Learning and Prompt Engineering to categorise, summarise and label (urgency level) complaints. This means that there is an inherent reliance on the general purpose data that is not visible to the council. Due to a lack of a clean available dataset, students had to resort to AI generated complaints to test their system, this needs to be addressed for obvious reasons. The council should look to build their own database of complaints categorisation, labelling and summarisation, such that the LLMs can be fine-tuned to match desired outcomes, and ultimately lead to more reliable and explainable behaviour from the AI model.

Scale and test: The system needs to be tested with the actual volume of complaints that the council receives to ensure it can handle the load effectively. This step is crucial for identifying any potential bottlenecks or performance issues. Can the system be effectively scaled to meet every council’s needs and prevent individual developmet?

Enhance features & integrate with other data sources: The students have identified some additional features that can be developed in the future to enhance the product, such as an expanded dashboard with more customisable tiles, a task-tracking login system for better complaint management, and automated updates for residents. Furthermore, there is an opportunity to integrate other databases such as ward boundaries, relevant stakeholders to location and type of issue. This will allow for even more sophisticated features, for example, if there are numerous complaints from a particular area and issue type relevant to a council member then they can be automatically and be able to respond immediately. This can be especially powerful in the case of disaster.

Implement data security: Implement advanced data encryption methods to enhance the security of resident data, ensuring compliance with data protection regulations.

Look at integration with council workflows: Ensure that the system integrates seamlessly with the existing workflows of the council. This involves training council staff to use the new system effectively and making any necessary adjustments based on their feedback.

Increase cost effectiveness: Work with the council to assess the affordability of implementing the system on a larger, and production scale. This includes exploring funding opportunities, potential collaborations with the private sector.

The students said…

Mathew Stevenson:

I thoroughly enjoyed working on this project for DG Cities - it was a new experience for me, thinking about how technology could help local government. My role in the project was building the user interface, and this gave me the opportunity to explore how best to present the information to council, which added an additional challenge beyond the technical challenge of building a web app; trying to build a clear interface that would be useful to all council members, regardless of previous experience with technology, was a challenge to balance with providing plenty of information – but a challenge I found very interesting, and I am happy with our solution!

I hope the web app we developed can help as a proof of concept for the council, broadly showing that technology can help make managing complaints easier and more automated! Beyond this, I particularly hope our solution highlights three key things:

(1) Getting automatic insights into the data (quantitatively via the dashboard and visually via the map) could be very helpful in identifying both problem areas, as well as areas where solutions are succeeding; action can be taken and lessons learned from this, much quicker than trying to spot these patterns manually from a spreadsheet.

(2) AI doesn’t have to be made the ‘front and centre’ feature of a tool when it doesn't need to be – you can leverage some great utility from it, as I hope we have, but it needn’t be shoehorned in everywhere.

(3) These solutions can be simple and flexible; our dashboard tile system makes it very easy for a client council to request specific insights, and these would be very easy to add in! This customisability means the council could get even deeper and more specific insights as they desire them, which is useful again for identifying problems earlier, which in turn means dealing with them quicker and happier residents.

The primary barrier to making our system production ready is the reliability of the AI summary and urgency; because this could have a real impact on people's lives and the responsiveness to their concerns, it needs to be reliable, consistent and accurate. It needs further in depth testing, which we didn’t have the time to do – however the performance from our small-scale subjective tests is promising. In the meantime, to help with this problem of reliability, we made sure to make the full original complaint easily accessible from the summary popup, so council members could still cross-reference with the original complaint.

Junyu Meng:

I thoroughly enjoyed working with DG Cities on this project and seeing our vision come to life. I mainly worked on the database in the backend, making sure all the relevant information is stored correctly to allow for our desired functionalities. Clearly displayed complaints, actionable insights and resident satisfaction were our top priority and I am pleased to see that evident in our end-to-end solution. Councils will no longer have to suffer from large backlogs caused by the current manual handling process, as efficiency would be massively improved. Resident issues can thus be addressed more timely, amending resident satisfaction.

Our proposed solution still requires a few more steps before it would be ready for production, including testing and ensuring the system functions well under the actual amount of complaint data that the council receives. However, we believe that both councils and residents would greatly benefit from a smarter complaint management system, should councils deem it affordable enough to implement it into their workflow.

A huge thank you to the team at Imperial - it was an impressive piece of work, with great potential benefits. Thanks to Bhavya Sharma, Matthew Stevenson, Junyu Meng, Ben Marconi, Alex Dhayaa and Sasha Afanasyeva.

We at DG Cities are working with councils on the many potential useful and ethical applications of AI and we’re committed to carrying this momentum forward into integration planning and initial testing and trials, so feel free to reach out if you are keen to collaborate or discuss further.