Google PLaces API (New) Securing in Flutter

In google cloud console when I try to lock down my api to specific Android or iOS app only for the calls, it always fails. It just seems to block everything. IF I open the code works fine, but I obviously want to secure it to only use the api key fro my own apps…

Am I missing somehtig simple here?