Speaking of Science: Evaluating 23 years of Dementia and Diabetes research

This month for Speaking of Science, we were joined by an expert panel to launch ‘An evaluation of NHMRC-funded dementia and diabetes research’, a report commissioned by NHMRC and delivered by Elsevier.

With a focus on research impact, Dr Luigi Rucco (Elsevier), Professor Josephine Forbes (Australian Diabetes Society), Professor Christopher Rowe (Australia Dementia Network and University of Melbourne) and Professor Steve Wesselingh (NHMRC) discussed how 23 years’ worth of research ultimately contributes to significant, lasting improvements in public health and societal welfare, aligning with NHMRC’s mission of building a healthy Australia.

Listen as we unpack the methods behind developing the report and the significance of the findings, as well as the impact of NHMRC-funded research towards 2 pressing health challenges faced by millions of Aussies.

Recorded on Tuesday 18 March from 10:00AM – 11:00AM AEDT.

Video transcript

0:00 Simon Mair
We're good to go. Excellent. Welcome, everybody. Thanks for thanks for coming, and welcome to this March edition of Speaking of Science.

My name's Simon, I'm the Executive Director of the Technology and Data Branch at NHMRC. Before we formally begin, I want to acknowledge the Ngunnawal people as the traditional custodians of the lands and waters of the Australian Capital Territory and surrounding region of where I am today.

Also, I’d like to pay my respects to First Nations elders past and present, their ongoing culture that connects and enriches our country and pay my respects to the traditional custodians of different lands on which we are all joining from today. This is very much a national audience online, so welcome everyone and a shout out to the Wurundjeri people of the Kulin nation from Melbourne, my hometown.

Before we begin, just a bit of housekeeping. There'll be an opportunity to ask our panel members questions towards the end, so please put your hand up in the room if you have a question. We have a roving microphone. If you're online, please make sure you are also on mute unless you are chatting, but there'll be a chat function where you can submit questions if you have any. One last reminder that our past recordings of all our webinars are available on our website and today's discussion will also be recorded.

This is a very special session. It's a little bit different this Speaking of science session because we're launching something, we're launching a report. We're very excited to have this report going live today.

Over the last 23 years, we've funded research into two of the most common and pressing health challenges faced by many Australians.

As of 2025, it's estimated that approximately 433,000 Australians are living with dementia. Additionally, an estimated 1.7 million Australians are involved in the care of someone living with dementia, including myself. Approximately 1.3 million Australians were living with diabetes in 2022, which equates to roughly 5.3% of the population. These statistics highlight the significant national impact of both dementia and diabetes.

On that note, today I'll share the privilege of launching An Evaluation of NHMRC funded Dementia, Diabetes and Dementia Research report.

This report commissioned by NHMRC and delivered by Elsevier, our guests today, provides insights into the research outputs, outcomes and impacts of NHMRC funded health and medical research in the areas of dementia and diabetes. We're committed here at NHMRC to fostering and promoting and evaluating the impact of health and medical research we fund. This impactful research holds many potential benefits and creates opportunities to ensure all Australians have access to evidence based authoritative health advice.

I'd now like to welcome our guests, and I'm delighted to introduce them. First of all, we have Steve as CEO, welcome Steve.

We also have Luigi, who I'll invite up to the lectern in a minute, who is the Head of Analytical Services for Europe and Emerging Markets. Luigi serves policy makers, funders and academic corporate research institutions around the world, providing advanced analysis on research performance by combining Elsevier high quality data sources with technical and research metrics.

I'd also like to welcome a few members of our guests. So, we have one in person. We have Josephine Forbes, who's the President of the Australian Diabetes Society. Welcome, Josephine.

Josephine from Mater Research-UQ in Queensland is the current President of the Australian Diabetes Society, advocating and supporting better and more equitable health care and research funding for people with diabetes. Her translational research extends from bench to bedside for novel therapies to prevent type 1 diabetes and kidney disease in vulnerable groups. She is part of the leadership group of the Australian Centre for Accelerating Diabetes Innovation and co-lead of their kidney disease programme. Josephine will join us on the panel a little bit later along with Christopher Rowe.

Chris, if you're online, if you're able to turn on your video and wave hello. Christopher is the director of the Australian Dementia Network from the University of Melbourne. He's a neurologist and nuclear medicine specialist at Austin Health. Good day, Christopher, thanks for joining us. And a professor at the University of Melbourne and Florey Institute of Neuroscience and Mental Health. He's also the director of the Australian Dementia Network, a national collaboration of 21 universities and research institutes.

Now at this point, I'd like to hand over to Steve who will introduce the key findings of the report ahead of handing over to Luigi, who will talk a bit more about the report and then we'll have the panel coming up. Over to you, Steve.

5:06 Professor Steve Wesselingh
Thanks very much, Simon.

I'd like to welcome everyone who's online and everyone who's in the room here today. I'm really excited actually about today and this report because for us as funders, it's really important for us to understand the impact of our funding. I think for a long time we have we haven't been able to do that and with this report, I think haven't quite got to the perfect outcome, but we've got very close to a really terrific outcome in understanding the impact of our funding.

I think in our conversation as a panel, we'll talk about how we can even improve that further and use more NHMRC data and more of the skills of Elsevier to understand the impact of our funding better. But I really think it's important that we, I understand why this is important and it's important because the community demands this of us. They actually give us the money that we use to fund this research, and the community needs to know how good the outcomes for that funding are and so I'm really excited by this report.

Simon mentioned that we have committed and spent about $679 million since 2011 on diabetes research and $622 million on dementia research. So quite a lot of money over a big period of time, but that that expenditure is comparable to our international funders. It again was important to look back over that period of time and choose a long period of time because that's how long it takes research to have an impact. If we look back over three or five years, we're not going to see the impact of our research.

When we look back over that period of time, what we found was that 9% of dementia and 15% of diabetes publications were cited in policy documents. This is where the research has gone all the way from discovery to changing policy, changing government policy, care policy and so on.

Actually, this surpasses our global comparatives. That's the other exciting thing about the report is it enables us to look at NHMRC funding and compare it with NIH funding or Wellcome funding or other comparators.

In terms of clinical guidelines, 4% of dementia and 8% of diabetes publications were cited in clinical guidelines, and this includes 153 clinical trials for dementia and 216 for diabetes. Again, it's really nice to see the funding for clinical trials then moving actually into clinical guidelines that impact and affect the way patients are managed across the country.

In terms of commercialisation, NHMRC funded research contributed to 44 commercialised or trademarked dementia interventions and 101 commercialised or trademarked diabetes interventions. Again, one of the things we're talking about right now across the country in terms of our new strategies being developed around research and development and also the National Health and Medical Research Strategy is how can we make sure our research, and our funding moves those ideas all the way to products and devices and creates companies. Again, with this sort of analysis, we can start to understand whether we're achieving that or not.

In terms of publications and bibliometrics, there's a lot of information in the report and I urge you to have a look at all of the bibliometrics in the report, but 3% of NHMRC outputs were dementia outputs and 4.2% of NHMRC outputs were diabetes outputs, twice the global average. Then if you look at the citation data, and there's a lot of data there, we see NHMRC being in that leading group with NIH and Wellcome and a couple of other funders.

I mean, it's really important to see that our funded research, our money is achieving the same outcomes as the people that we look at as leaders in the area like the NIH and Wellcome. In terms of interventions and NHMRC supported research has contributed to 444 distinct dementia interventions and 490 diabetes interventions since 2000. A lot of really important interventions.

In addition, was cited in over 1000 patent families in both diabetes and dementia research, really showcasing the innovative nature and influence of the funded research.

In addition, we can through this process and through the fact that we're using both NHMRC data and then bibliometric data and large language models, we were able to see products that were developed through the research that was funded by NHMRC. We can see really important products such as Cogstate Brief Battery, a really important tool in Alzheimer's research and PromarkerD, which predicts rapid renal decline in diabetes patients. Both they're just examples of some of the products and tools that have been developed on the back of our research.

Again, you know, without this sort of analysis, which we'll talk about as in the panel, you know, NHMRC doesn't have the visibility of what that is and how our funding is or isn't helping for the development of those sorts of tools and that's really important to us.

Really we also needed to look at whether our research was funding research that was at the cutting edge of technologies such as artificial intelligence, bioengineering or gene editing and, and really understand were we finding the most innovative and was that comparable to our competitors, but also collaborators across the world.

That sort of information can then influence the funding that we do in the future and also how we report back to the people who fund our research, that is the taxpayers and the government.

That's just a few, I guess a little a smattering of what's in the report. I'm really excited by the report and also really interested to hear the discussion or be part of the discussion actually in a little while. But I'll hand back to Simon and look forward to the rest of the presentation.

12:02 Simon Mair
Thank you Steve. I’ll now welcome Luigi up to the lectern and to present the findings from the report.
Would you like some water?

12:10 Dr Luigi Rucco
Yeah.

So, thanks Simon for the presentation. Thanks Steve and thanks also for having invited me in this beautiful city.

For us coming from abroad, we normally think of Australia in terms of Sydney and Melbourne. But Canberra is a city to visit because here you can feel the good, the positive energy of the institutions of this country that really tell we care about our people. Yesterday we visited the Parliament and this openness, this transparency that embeds really into the very essence of the institutions is unique. It's fantastic, you should be really proud of your country.

Today I will present some highlights of the comprehensive report that the NHMRC commissioned examining 23 years of research on dementia and diabetes and this timeframe, this long timeframe allows us to capture economic and societal impact, which often take decades to manifest, while maintaining topical relevance. The report differentiates between NHMRC funded research and other international benchmarks.

Maybe we can go to the first slide. We combine established indicators with emerging ones to measure research translation, commercialisation, uptake in policy documents, clinical guidelines and clinical trials, as well as enabling factors like intersectionality, interdisciplinarity, participatory designs and other next generation metrics.

We employ a quasi-counterfactual design to robustly assess the contribution of NHMRC funded research versus non-funded research using self-controlled groups of researchers to account for confounding factors like career stage and speciality among others.

We analyse normal or improved health interventions, assessing their impact on public health and societal welfare, and this includes pharmaceutical measures, diagnostic tools, medical devices, behavioural interventions, health system announcements and others.

Using a human validated experimental AI approach, we provide enhanced understanding of our research leads to significant and lasting improvements for the economy, the society, environmental and health factors as well as cultural cutting-edge knowledge areas. We can go to the next slide.

Here we dive into the research outputs which are the direct products of NHMRC funded research in the fields of dementia and diabetes. Firstly, publications are a key research output.

NHMRC has funded 2762 publications in the field of dementia which account to about 3% of the total research funded. For diabetes, NHMRC has funded 3834 publications which account to 4.2% of the total output.

NHMRC specialisation in these fields out of the total research funded by the institution is about the global average and surpass other founders. In terms of citation impact, NHMRC have shown strong performance and place among the leaders of international funders along with Wellcome Trust, NIH, Alzheimer’s Association and others.

However, there is room for improvement, especially for data and code sharing, considering that only 15% of the research produced by NHMRC has code and underlying data published in other repositories, which is behind other funders. NHMRC research translation is not working with a significance using patents and it is on par with other global leaders in terms of patent citations.

Since 2000s, NHMRC has contributed to the development or evidence collection for 445 dementia interventions and 490 diabetes interventions, which include new pharmaceutical measures, diagnostic tools, medical devices and behavioural interventions. We can go to the next slide.

Here we explore the outcomes of NHMRC funded research, focusing on the tangible benefits and changes derived from the use or the application of the research outputs in dementia and diabetes.

Starting with economic outcomes NHMRC funding has contributed to building the evidence base for the commercialisation or trade marking of 44 interventions for dementia and 101 interventions for diabetes.

NHMRC also supported at least 12 startups by funding research that generated evidence or conducted evaluations or tested the products and services of these startups.

NHMRC funded research has significantly influenced policy and clinical guidelines, as we will see in the following slides, demonstrating greater uptake than global peers.

When it comes to media outreach, the dissemination of NHMRC funded research has varied. However, the NHMRC Boosting Dementia Research Initiative, the BDRI, has achieved significant outreach with 3/4 of the publications mentioned in online media and this announces public awareness and engagement.

Overall, these outcomes show how NHMRC transcends the academic boundaries influencing economic growth, policy development, clinical practice and public awareness. We can go to the next slide.

Here we see in particular the significant of policy impact of NHMRC funded research. Specifically, 9% of publications related to dementia and 15% of the publications related to diabetes have been cited in policy documents. You see in the first column, these are the performance of NHMRC, and this high rate of citation underscores the relevance and importance of the supported research in shaping our policies, and you see also NHMRC is above the other global funders.

To understand the true value of NHMRC contribution, we conducted the counterfactual analysis, and this analysis helps us to determine what might have happened in the absence of an NHMRC funding. The results indicate that the policy citation achievements that we see today would be unlikely without the support of NHMRC. This means that NHMRC plays a crucial role in ensuring that research findings are translated into policies that benefit public health. OK, go to the next slide.

NHMRC funded research has shown impressive citation rates in clinical guidelines with 4% of dementia related publications and 8% of diabetes related publications being cited. These rates are higher compared to other global peers and indicate the significant influence of NHMRC supported research on clinical practice. Furthermore, NHMRC outperforms major international organisations like the NIH, Wellcome Trust and others in this regard.

This demonstrates that NHMRC funded research not only contributes to scientific knowledge but also plays a critical role in shaping practice and improves patient care in both dementia and diabetes. We can go to the next slide.

On this slide, we'll explore the pathways through which NHMRC funded research leads to significant and lasting improvements.

Firstly, let's consider the knowledge impacts. These are cutting edge areas where NHMRC funding contributed to advance the research, especially in fields like bioengineering, gene editing, patent targeting, AI in healthcare, prevention strategies and place-based prevention, pre-disease care, effects of climate change and health, cultural factors and health disparities, health systems and urban planning is all, let's say the characteristics of the knowledge impacts that were captured by our analysis and where NHMRC contributed.

Then we have another type of impact stories that relate to economic, environmental, social and health impacts. In this category we include aspects like health economics, health equity, health services, hospitalisation rates, prevention and other wellbeing aspects.

Another approach leveraging AI, specifically large language model has been employed, and this allows to really explore vast amounts of literature and uncover hidden patterns, trends as well as correlations.

Importantly, all results have been manually validated and created to ensure accuracy.

In terms of knowledge impact, more than 25 case studies have been uncovered. Regarding economic, environmental, social and health impacts, 17 notable case studies have been identified and there is also substantial material for additional case studies in future. We can go to the next slide.

Here we show a little bit the methodology of our AI based approach. For knowledge impact, we use a large language model to identify cutting edge research areas influenced by the NHMRC funding. We classify NHMRC funded publications into this thematic data sets that represent the cutting edge areas and then generate factual impact summaries representing these contributions that are then manually created and validated by the team.

For economic, societal, environmental impacts, the process is slightly different, and you can see that in the method diagram of the slide. We first identify the publications that cite NHMRC publications, then the LM tags the citing publications according to the respective impact captured, can be health economics, can be hospitalisation rates reduction. Of course, there are citing publications with no impact come captured and those are disregarded. Of course.

Then we cluster related publications, and we create summaries by extracting quantitative evidence like statistical outcomes and effect sizes. The summaries that are factually and evidence based are manually validated and curated by our team to ensure accuracy and also that the proper context is captured. This hybrid approach combines the efficiency of AI, which is able to work at scale on past large amounts of literature with human oversight.

You can go to the next slide, which is an example of a knowledge impact case study. This one related to dementia, and in particular this one is about the use of machine learning to predict and diagnose the mild cognitive impairment, which is at an early stage of the Alzheimer's disease.

There are several studies studying in this case study, but basically ML machine learning has been used to detect certain subtypes of MCI for example amnestic MCI by looking at the brain imaging, the changes in the brain structure or even the white matter alteration.

There are other studies that distinguish between different types and subtypes of MCI by looking at biomarkers imaging and revealing different brain thinning patterns that can allow distinguishing between the different subtypes of MCI. In addition, this AI based methodologies for the diagnosis of MCI combined it's only brain imaging but also the result of cognitive scores.

The next slide is about an economic social impact case study. This one is about the SGLT2 inhibitors that are used to reduce hospitalisation rates and improve renal outcomes for diabetes patients but also other classes of patients. In particular, this case study talks about certain family of inhibitors of drugs that reduce blood sugar by causing the kidneys to expel the sugar through urine, and these drugs are Pioglitazone, Canagliflozin and the Dapagliflozin.

The studies cited in this case study demonstrate that these drugs can reduce hospitalisation rates by certain percentages but also reduce the risk of cardiovascular diseases, as well as improve the renal outcome by reducing the risk of dialysis or transplantation or even let's say the worst consequences of these diseases.

We can move to the next slide where we provide a few recommendations for future studies.

First of all, we suggest building a sustainable data pipeline anticipating the need for extensive custom data cleaning and processing, and this can be possibly integrated in future studies and evaluations.

Secondly, we suggest developing an investigator self-reporting system, encouraging researchers to self-report outputs, outcomes and impacts and to reduce the burden of this reporting, you can even provide some retrospectives follow up about what's the past, let's say performance in terms of outputs, outcomes and impacts the researchers for them to draw on.

Third, scale up AI based impact discoveries. At this stage of the technology, human oversight is still very important, but we see an acceleration in the accuracy of these models with reasoning models, identity workflows. So in future, we'll probably be able to scale up this methods to really almost the entirety of the outputs produced by the funded research.

Last slide, I would like to thank you again for I mean inviting me. Credit goes to my colleague Etienne, who was the technical leader for this study as well Dimitra, Elizabeth Browning, who participated among the others.

This is just, let's say, the summary of the report, which is under you to read the report, where we also have a detailed overview of the methodology and a deep dive on the findings. And I leave the word to Steve for the next session.

Thanks again.

28:54 Simon Mair
Thank you very much, Luigi, and thank you to Elsevier for doing that amazing piece of work.

I just find it fascinating, the things that AI can do. We're learning more and more about it every day. It's exciting and also a little bit confronting.

We'll move now on to the panel session. If I can invite Josephine, Steve and Luigi up here to the chairs that we have, You can fix yourself a water if you if you need. 
I think we have some microphones for you to speak into and we have a roving one for any questions in the audience online.
If you have any questions online, please put it into the chat and Alyssa will read them out to us. But just to kick us off, I've got a couple just to get us warmed up.

I might start with one for you, Steve, just around the background behind this piece of work.

Why is this something that's important to NHMRC to do this analysis around outputs, outcomes and benefits of our research in a different way to something we've done in the past?

29:59 Professor Steve Wesselingh
Yeah, so really important question and I guess it goes to what we've measured previously.

We've been, I think really good at measuring where our money goes, who it goes to.

30:11 Simon Mair
I'm not sure if the mic is working, just check that.

30:17 Professor Steve Wesselingh
Maybe I'll start again.

A really important question. I think in the past we've been really good at measuring who we how, how we fund people. We look at our peer review carefully, we measure who we fund, and we look at the categories of people we fund, whether they’re early career or late career, we look at gender, we look at all a whole lot of areas. We've been very good at looking at that.

But once we fund people, we haven't actually been quite as good at looking at what's the output of our funding. And obviously I think that's critical to people who give us the money to fund, the governments, to the health portfolio and so on. I was really looking at a way to measure the impact and I know impact is a word we utilise quite a lot these days, but you know, what's the impact of our funding?

Then the opportunity arose to work with Elsevier and then we started thinking about it. Then the idea to look back 20 years was really exciting, and then to look at 2 areas which really have an enormous impact on Australians. Diabetes and dementia, really common diseases, but also diseases that change families, you know, enormous impact and an impact that's only going to grow, unfortunately. So we chose those two areas.

We also chose dementia, not only because it's such an important disease, but because we had had the dementia initiative. And so we had allocated some extra funding in the past to dementia research and we thought it was important to have a look at that had achieved.

32:03 Simon Mair
Excellent, thank you very much, Steve.

I might go now to Josephine and Chris.

Working in those sectors, working in those areas, and some of the challenges, if you wouldn't mind highlighting that you face around evaluating the impact of research and if you have any feedback or observations on the report and the piece of work that we've done and what it means for your organisations?

I might start with Josephine first in the room.

32:29 Professor Josephine Forbes
So firstly, we're very thankful to the NHMRC and Elsevier for the opportunity to show how much impact the investment in diabetes research has actually had because that's particularly, we have known that, but it's really nice to have numbers to actually show that.

That's the first thing that I would like to say.

I think as to the limitations of what has come out in the report, I think the first thing I'd like to say is that it doesn't represent the views and needs of people living with diabetes. I think it would be very nice to understand how to capture and put those impacts forward as well as part of this type of report.

I guess one of the other things I'd like to highlight is that it also doesn't separate some of the vulnerable groups who are overrepresented by diabetes. That includes people who are First Nations who have about 13% incidence of diabetes compared to 5% and also younger people. Younger people are increasingly getting diabetes and so it'd be really nice to actually separate those groups and see if we are particularly having impact on those.

I guess thirdly, one of the other limitations we can see from the report, which as I said we're very delighted by, but we also can see that commercialisation and spin out companies is also something that is underrepresented from diabetes. The reason from that is because we have lots of spin out companies and they are certainly things that we can identify, but our research gets hijacked for other areas and the reason for that is because it's very difficult to translate things into clinical practice in diabetes. We have to do very large cardiovascular trials. That scares a lot of companies and so they think we'll go for something that's a little a little bit easier.

I think understanding how to facilitate those findings, staying in research for diabetes and then translating into these commercial and then clinical outcomes, is also something that we would like to understand how to do better.

34:27 Simon Mair
Thanks a lot, Josephine.

Chris, over to you.

34:31 Professor Christopher Rowe
Fascinating report and lots of bedtime reading in the whole 130 pages. But it's reassuring that the NHNRC is clearly on track by world standards in terms of the productivity. Well, sorry, the outcome in terms of citations and publications etcetera from the investment that they make.

It has highlighted how steep the funnel is for research, and we start off with so many publications but end up with, particularly in the dementia field, so few patents. It's given us metrics that we can work towards, we can aim to improve our productivity, and I think that's extremely important.

It does show that dementia researchers are media tarts compared to their diabetes colleagues, we seem to have a very high media publicity for our papers compared to diabetes. They're clearly more studious.

I think it's very useful. I agree that it would be good to get some of these metrics included in future grant applications. It is a good way to evaluate the impact of researchers. I think a lot of that data is probably already there in the track record statements but could be pulled together in a slightly more productive fashion to give us a real impact on previous grants that investigators have had.

Dementia does have a particular difficulty in terms of producing products in terms of therapeutics. For 20 years, we haven't had a successful new drug, and I think unlike diabetes where they do have lots of treatments to play with and refine and work on, we've been a little bit bereft in the dementia field in terms of therapeutics to work up and improve. Hopefully that is in process of changing now.

A lot of dementia research has been quite care focused. Again, that is not something that leads to patents or startup companies etcetera. It's very important, but it's something to keep an eye on that we need to keep a balance of productivity in terms of treatment and things that are beneficial to our economy while maintaining the investments in care research.

Overall, a very interesting read. I think there's something in it for anybody. It's potentially a bit of political dynamite, depending on what people pick out of it and emphasise. But if you sit back and look at it overall, I think it reflects well on the NHMRC and on their processes and on the productiveness. Well, certainly the citations and publications that have been coming out of NHMRC funded dementia research.

The other thing we've got in Australia that's a bit odd that other countries don't have is that we've got the parallel funding stream of the MRFF. So if you put that together with the NHMRC I think our dementia research output would be even higher compared to international funders such as the NIH. That's it.

38:06 Simon Mair
Excellent. Thank you very much, Chris and Josephine for your comments.

We might throw it open now for any questions from the audience either here in the room or online.

Is there anything coming through online? Yeah, Alyssa.

38:30 Secretariat (Alyssa)
Is MRFF-funded research included in the analysis?

38:34 Simon Mair
The question was, is MRFF-funded research included in the analysis?

That's something possibly Steve can start with and Luigi you might have comments on.

38:44 Professor Steve Wesselingh
Yeah, I think that's a really important point that Chris and obviously the question brought up and people are aware, we're moving closer and closer to the MRFF and bringing NHMRC and the HMRO closer together and with the goal of harmonisation.

I think that would be a future question.

The answer is MRFF is not in the analysis, but we do have a data group that's bringing the data of the MRFF and the NHMRC closer together for analytics. I would be hopeful that the next time, as we progress this and look at it again, that we would include MRFF funding because that would just make a lot of sense.

39:26 Simon Mair
Yeah, absolutely and the information that we provided to initiate the analysis that Elsevier ran were not difficult to obtain. It's just a matter of going to our data warehouse and going for the looking through all the grants that were ordered over the past 23 years. That's something that we could quite easily approach HMRO and get the MRFF data for a future piece of analysis. So we'll certainly do.

Luigi you did you have any comments on any of the opportunities around if we were to provide more of the grant information to start that analysis, has that changed the way that you would do the work or run the analysis piece?

40:05 Dr Luigi Rucco 
Sure. I mean, this is the green.

Yeah, sure. First of all, in the report, we indeed consider other Australian funders. They are aggregated and it's just like an overall benchmark with which NHMRC is etcetera.

But of course, having data available would allow us to also perform similar analysis for MRFF. I would like to add something because now we are always talking about public funders and of course in this regard, NHMRC is among the world leaders. Is for sure a member of the Six Nation League if we want to, I mean make a parallel with the sports.

But of course, there is a big player which is not in the room, which is the private sector. Most of let's say the impact coming from the private sector is, is huge. If we compare the private sector with the public sector, then we can really observe significant discrepancies. So probably looking forward to a future exercise. I would include some say parallel benchmarking not only with public funders, but also between the public sector and the private sector. Now they are cooperating because the impact coming from the private sector is probably the highest among all other players.

41:33 Simon Mair
Thanks, Luigi.

Alyssa, do we have any other comments or questions online?

41:38 Secretariat (Alyssa)
I'm interested in the time taken to undertake the whole review and feasibility of doing this type of review regularly to assess impact for other research areas.

41:49 Simon Mair
Excellent.

I'll paraphrase that one. So just for the audience online, so you can hear it, you might not have picked up Alyssa over the microphone.

This is probably a question for Luigi just around the length of time that it took to carry out this process and this analysis.

42:03 Dr Luigi Rocco
Well, this particular project has been very R&D intensive. We really put a lot of technology and brain power in this. It takes time both to perform this kind of analysis, but also it takes time to see this impact manifesting.

There are two different times we're talking about here. The time we analyse the 23 years of research in this space, because we need a longer timeframe if you want to capture the long lasting improvements for health and society etcetera.
But also, it takes time to perform this. My view being in this sector is that AI is changing the rules of the game. AI can really perform accurate analytics at scale and can crunch huge amounts of data of different sources, finding patterns and let's say correlations that we don't see.

At the moment we still need a lot of human oversight, but this is going to change because we have now these DeepSeek models. Everyone knows that the renew release by AI, but also DeepSeek has been recently chumped on the headlines and alongside reasoning models, we also have identical flows. Identical flows are models by which I am an AI agent specialised in, for example, analysing literature. Then I have another agent specialised in analysing patents, and other agents specialised in analysing clinical guidelines and they talk to each other. Then we have other agents reviewing the work done by this to ensure accuracy and avoid hallucination and so on and so forth. That kind of human interaction and entering in a dialogue among themselves to produce this kind of reporting.

The future I see is this agency workflows working in autonomy, crunching huge amounts of data coming from different sources and producing this kind of analysis in a very efficient and timely manner.

44:09 Simon Mair
Thank you very much, Luigi.

I can comment as well if I may on the customer experience working with Elsevier.

The work that the work Luigi and his team did was incredibly fast. In fact, so fast that it exceeded the speed with which we can negotiate the contract to actually engage Elsevier to do the work with us. So yeah, it was very, very, very rapid and I'm sure overtime with the advances in the technology and the maturity in your team, Luigi, it will move even faster in the future.

Josephine.

44:46 Professor Josephine Forbes
If I could just add a quick comment and again, Chris, it was suggesting something that you said.

We've had a lot of new therapies and a lot of new technologies coming to the diabetes sector.

This type of report will be incredibly useful and helpful to us understanding the impact of those and I guess the areas that are not being addressed by those particular therapies and things. We're also thankful for that benchmarking.

45:07 Simon Mair
Good point.

Thank you. And Steve?

45:08 Professor Steve Wesselingh
I just wanted to make in the private sector, I think it's a really important point and it's both the philanthropic private sector, but also obviously the private sector that creates devices and drugs and so on, big pharma as an example.

But I do think again, these sorts of analyses are really important because I do have a bit of a bug bare about this. But most of the things that the private sector ends up producing, including big pharma, including Tesla and including Apple comes from the public sector. The research is done in the public sector moves into the private sector and that's their job is to turn it into a product or a drug. Again, this sort of analysis, I think would be really exciting to have a look at that and look at how much of the factory is in the universities and the MRIs for our case in Australia, but also around the world and that then moves into the production of wealth in the private sector, which is what we want to do. And that and obviously we want those products and drugs to make help patients. But I just think that process is a really important one and I think you can only start to see it with these sorts of analytics.

46:21 Simon Mair
Yeah, really important and extremely complex to understand and, you know, achieve a result at the end of the day, you know, to take 23 years for something to realise or the length of time it can take for some research to realise value. It's very different to some of the things I'm familiar with where you have a project and you want immediate impact but having to have all these different players play that critical role to get to that value at the end is a really complex beast.

We've got a question in the room I think if we can get a microphone over there.

47:02 Audience member
Thanks. My question is in relation to the clinical guidelines that you mentioned that there's impact with, is that just in Australia those clinical guidelines?

And my second follow up is whether the impact that you've measured have our international impacts, not just Australian, I'm not sure if you said, but I just wanted to ask.

47:22 Simon Mair
So this is the reference to the clinical guidelines in the Elsevier slide.

47:28 Audience member
You mentioned some uptake in clinical guidelines, and I was wondering is that just in Australia those clinical guidelines?

And then my second follow up is whether the impact of our research and citations is international or just the measurements are from Australia.

47:42 Simon Mair
Is that something Luigi you're able to comment on with your comment on the clinical guidelines of the presentation.

47:43 Professor Steve Wesselingh
Maybe Chris and Josephine should comment on the reach of Australian clinical guidelines. But then yeah, the citations are international. But Luigi, you might want to comment on that.

48:00 Simon Mair
Yeah, that's true.

We might go online first because we heard from Josephine a minute ago. So Chris over to you for that question.

48:06 Professor Christopher Rowe
OK.

As far as I know this includes international guidelines in which NHMRC researchers were mentioned.

That does give us a disadvantage though in Australia in that when we're on committees, international committees, they're usually multi author, they're usually run from by somebody in the United States or in Europe and the NHMRC may actually miss out on acknowledgement. I think we do need to encourage our investigators to always make sure they acknowledge NHMRC and that does get acknowledged when there's international guidelines put out.

But so, the impact on policy and guidelines, I think has been international as well as national.

48:55 Simon Mair
Thank you, Chris and Josephine.

48:56 Professor Josephine Forbes
I'm just going to add that that's absolutely true for diabetes as well.

We're more and more working with the European Association for Diabetes and also with the American Diabetes Association to put out joint position statements. But we do need to make sure that NHMRC is acknowledged because I do agree that sometimes Australia is the small fish in the room and we need to have a very loud voice.

49:17 Simon Mair
Thank you.
I know the acknowledgement of the guidelines that NHMRC produces is important as is the acknowledgement of the grant funding for the researchers that that is produced. So one of the things that Luigi and Etienne and their team had to do is to go through and find all these artefacts to determine did a grant result in research? Was that research funded by NHMRC or not?

There were a number of things where NHMRC was cited as the funder for the research, but there were also a lot of papers that were found where we probably were, but we're not listed. So that's another message and a recommendation, for everything that we fund, we are really interested in having that and NHMRC acknowledged as well as a funder of that research, so we can do more of this analysis and understand the impact.

50:04 Audience member
I have a comment.

50:07 Simon Mair
Yeah, we have a comment as well from Cass in the room from Elsevier.

50:13 Audience member
I think that's a really interesting statement to make the future of this kind of research. I can talk from my own personal perspective.

When we looked at that knowledge impacts case study, it's really interesting when we think about the advancement around MRI capabilities and exactly what we're going to come across as someone who suffers from a neurological disease such as MS. I think the translational research and research impact moving forward will be really interesting to look at that space because while we're looking in this respect at Alzheimer's disease in particular, we can also then think about that translational research impact on other neurological diseases. There's a huge scope here to understand that impact, and I'd really love to hear a little bit more about potential ideas in that space moving forward.

51:04 Simon Mair
Thank you, Cass for that comment. I'm not sure, Steve, is there anything?

51:07 Professor Steve Wesselingh
Yeah, I think that's really important. I mean, we obviously have to, otherwise the project would have been would have been huge if we'd just chosen all NHMRC research.

But your point is well taken that someone might well work on a drug for diabetes but actually find that it affects another area. And in fact, a lot of diabetes drugs are now being looked at in terms of dementia. So they actually are in these two areas, but there'd be multiple other areas like cardiovascular disease and so ultimately, I think yes, it would be nice to look at all of our funded research because there's so much intersection.
In fact, we looked at multidisciplinary and interdisciplinarity here and I think that has to be a measure and it's clearly a strength. But the more areas you look at, the more you're going to see that. I don't know whether you want to comment on that, Luigi.

51:58 Dr Luigi Rocco
That's absolutely true. We observed the same in other scientific domains, so it's becoming more and more translational.

This will, I mean, not only impact the out research, but we'll probably see a look for these kinds of patterns also by analysing other disciplines, you know. AI, for example, AI is having a disruptive effect on how we are taking diseases, diagnostics and prevention and it's coming from the computer science world, just to make an example. But there are others.

So fully agree, it will have even more prominence in the future studies for sure.

52:39 Simon Mair
Thank you, Luigi.

That probably brings us nicely to time. I think first of all, I'd like to thank everyone for attending online and also in person.

I'd also like to thank our guest speakers, Josephine and Chris for joining us on the panel and also Elsevier for helping us with this report.

Steve, would you like to make some closing remarks before we before we wrap up?

53:05 Professor Steve Wesselingh
Yeah, but I actually just wanted to check that Chris and Josephine didn't want to say something right at the end if they had any final comments or Luigi.

53:13 Professor Josephine Forbes
Yeah, I just wanted to reiterate our thanks for the opportunity to show the impact of diabetes research. Thank you.

53:23 Professor Christopher Rowe
No final comments from me, but thanks for the opportunity to review this this work and give me a greater insight into what dementia research has been achieving in Australia.

53:37 Dr Luigi Rocco
Yeah, on my end, I serve customers and governments all around the world, really, so almost all countries. What we have done here is unprecedented. The level of R&D and innovation that has gone into these reports, in my view, we change the way we do evaluations from now on. NHMRC has been a perfect partner to really explore new ways of conducting research assessment.

Thanks a lot.

54:05 Professor Steve Wesselingh
Thanks for that.

From my point of view, I think it's been fabulous.

I have always believed that we need to do more analytics in terms of what I call research on research or meta science, understanding the impact of our funding and understanding of right along the pathway all the way to patients. I think your point was a valid one, all the way to patient groups and patient advocacy and I think this is just a start, but it is a really powerful start.

I'm really keen to, you know, develop more capacity inside HMRC in terms of developing our data so that we can then use the tools that are available to really analyse it and scrape the web to look at the impact of our research. Then you know, talk to our community, our politicians, our patient groups, our advocacy groups about the data and how it informs us to do what we do better, and that is to fund great research.

55:10 Simon Mair
Thank you, Steve.

Thanks everyone for joining our March Speaking of Science webinar.

It's always a privilege for an HMSC to acknowledge the impact of the research we fund and how it's helping achieve our mission in building a healthy Australia.

We didn't mention earlier that that both the report and the infographic will be published on our website and on the Elsevier website very soon, so please log on and read it. Probably don't read it all in one sitting, it's quite a lengthy report, so maybe take a few nights to get through it.

But I'd like to thank Elsevier especially for their hard, hard work and Luigi, we know you travelled a long way to come so thank you very much for coming in person. That's greatly appreciated.

Also thank you to our panellists and everyone that join us online in the room. We look forward to seeing you at the next Speaking of Science session. And we also look forward to having further discussions about this piece of analysis and the approach that we're using.

Thank you very much.