Currently in preview, you can now get some usage insights of Azure AD Application if users are authenticated by Azure AD only; this includes Azure AD App Proxy applications.
To get these insights, logon to your Azure portal (https://portal.azure.com) or Azure AD Portal (https://aad.portal.azure.com) and reach out the Enterprise Applications blade and then Usage & insights
The Usage & Insights is providing you a report of the most active application within either a 7 days or 30 days period
For each application you have the number of successful and failed sign-ins. You can then go to a specific application details by clicking on the application name, this will redirect you to the application configuration blades.