IMPATIENT SERIES #5 – EVALUATING AND TRACKING PROJECTS

In the first entry of the Impatient Series I discussed what patient organizations mean when they say they are doing research. 

This entry adds to Impatient Series I and to the ImpatientRevolution book and discusses in more detail how to determine that a proposed project is the right one and how to monitor its progress.

When you let the scientific community know that your organization is open to fund research around your rare disease you are likely to get many research proposals from academic groups. These will come in different qualities, and will have different relevancefor your disease. 

As in the proverbial “if all you have is a hammer, everything looks like a nail”, most groups will try to tell you that the way to address your disease is to apply exactly the same techniques that you are using to your particular disease model. This is not an attempt to mislead you, since they all have chosen to build their laboratories around that specific technology or approach because they believe in it. This is why you need impartial advisors to help you select the best and more relevant applications.  

HOW TO JUDGE THE QUALITY

The best way to determine the quality of research proposals is to use some trusted scientists as reviewers. Scientists are used to evaluating other scientists research proposals and manuscripts, it is part of their usual jobs. 

Some of the aspects they should pay attention to are: does this group have the expertise that they need for this project? Are the proposing the right controls? Is the amount of work realistic with the proposed duration and funding request?

I also like to ask if this is the best lab that could do this approach, since a group might conclude (perhaps correctly) that the best approach for your disease is to develop a gene therapy and propose to do this in their lab, even though they are not a lab that has done gene therapy before. 

HOW TO JUDGE THE RELEVANCE

It doesn’t matter how excellent the research proposals are, if they don’t directly advance your agenda you should not fund them. You did not create a Foundation to compete with the NIH for funding the very best grants, you created it to fill the gaps of your specific disease research field so that therapies can be developed faster.

Because of that your scientific advisors should also be able to evaluate if a given proposal is an interesting area to explore, vs a core bottleneck in the field to address, to cite to extremes. 

To make the life of the applicants easier, I recommend that you decide, before opening a call for grants, which are the core interest areas that you are seeking proposals for. The more specific you can be the better. Saying that you are looking for proposals to “advance knowledge of the disease X, translational research or clinical research” would be too vague and of little help. Saying that you are looking for proposals to “identify substrates of kinase X, develop translational biomarkers and to validate new clinical outcome measures” would be much more clear.

Having pre-determined your key interest areas will also help you see if there is one of them for which you are not receiving proposals, in which case you might want to directly reach out to specialists in the area and propose them to work with you to address that need.

HAVING AN INTERNAL TEAM

At this point, if you are a young patient organization or foundation, you might be wondering how to manage the complexity of identifying scientific advisors, sourcing grant proposals, and determining research priorities. As I explained in the Impatient Series I entry, I am a strong advocate for you to consider having a researcher or an internal team, depending on your side, that works for your organization and that can be on top of all this. Having a researcher will also facilitate conversations with the industry and with regulators, since the right researchers will have a background that will make them capable of interacting with those professionals. 

Having a scientist that works with you, even if part time, will also be very important for monitoring the progression of the projects that you are funding and making sure these are successful. In the next Impatient Series article I will discuss my thoughts about scientists working inside patient organizations or foundations.

MONITORING PROJECT PROGRESS

For monitoring project progression, I recommend quarterly project reviews (ideally written reports), and a final written report. Unless there is a big milestone coming up soon, requesting updates more often than that is not useful for research. Of course you should always have back and forth of short communications with the labs to make sure they all have what they need to move forward and be able to help with the troubleshooting.  

It is a good practice when you follow up with the labs to ask “is there anything we could do to help you with this” whenever the lab expresses some delays or difficulties. This is also why it is useful to have a scientist doing these follow ups.

At the Spanish Dravet Syndrome Foundation we also used to run twice a year portfolio reviews, which were a half a day face-to-face meetings where every funded lab would present their results in the last period. At the Loulou Foundation we do this once a year, also for half a day, the day before our big CDKL5 Forum. The benefit of these meetings is that not only the Foundation gets an update about the projects, but all the different groups in your current portfolio get to see each other’s presentations and every single time that starts sharing and collaborations. I find these meetings very useful. 

If you schedule these portfolio reviews right before another important meeting, for example your annual families meeting, and ask the groups to budget for this trip in their grant proposal, you will make it easier for them to also attend your meeting and potentially give an update to the families and other members of the community during the same trip.

At the end of the funding period, the groups should provide you with a financial update, outlining where the money was spent. For quarterly reports I think only a scientific update is needed. 

Another important aspect to consider is intellectual property (IP). Some organizations don’t ask for any rights to part of the IP, while some others might ask for too much. I like very much the Loulou Foundation policy of asking for a percentage of the benefits from IP proportional to the funding that was used to create that IP. This empowers us to ask the groups to consider if any part of their discoveries is susceptible for IP creation, for example the method of use of an old drug for treating your disease if that is what they tested in their project. In this specific example, an academic group naive to how drug development works might publish the results without submitting a provisional patent application first, making it impossible for a company to then take that drug into clinical trials for your disease. Having the rights to part of the IP revenue also gives us a seat at the table during technology transfer negotiations, and help us see if the university is making enough efforts to get that IP turned into an actual product or drug program or if they are sitting on it and not making any efforts. To prevent this scenario, you might want to include some clawback provision into your intellectual policy clause which the groups need to sign in order to receive your funding. 

Although asking for shared IP rights is attractive, you also have to be realistic. There are very few CureDuchenne. If you can only give small grants, you can’t expect to have shared IP. For example most patient groups I. know can only provide $20-50,000 per project, which is just co-funding the project. One the other hand, many of the Foundations are able to give significantly larger grants, of over $100,000 per year, and stand a more realistic chance of asking to have a percentage of the IP revenue. To know what you can ask, I recommend you seek advice from people familiar with technology transfer who will be able to advice you based on the particularities of your organization.

Last, make sure you request your grantees to notify you of any publications and conference presentations that result from the project that you fund, and that they include your funding in the acknowledgments. This will help you track the results of your funding beyond the duration of the funding period. 

 

The next entry of the Impatient Series will address how to find a scientist to work with your organization.

Let’s start an #ImpatientRevolution!

Ana Mingorance, PhD