In order to preview videos across a wide variety of formats when uploaded to the platform, you will need to have the option to Convert Videos For Online Streaming enabled under Admin > Settings. This will ensure videos are automatically converted to a format that is then supported for streaming across any browser.
When this option is disabled, only videos in mp4 format will be supported for preview.
Note that the conversion is initiated on upload, or when the preview is loaded on a user screen if the conversion is missing. A delay should be expected between either of these actions, and the video being ready for only preview.
You can also enable adaptive streaming to create different versions of the video optimized for different screen sizes, internet speed and resolutions.