The OneDrive client has been updated and now provides access to the file version history from both the OneDrive client and Windows Explorer (for files synced with OneDrive).
You need run the OneDrive client version 20.052.0311.0006 or later to enjoy this new capability.