Steps in order to enable popup:
1. Install and activate https://wordpress.org/plugins/iw-magnific-popup/
2. Create custom menu link with some dummy url like this http://screencast.com/t/Ujr1LGmiur
3. Preview your page in google chrome and inspect it with developer tools in order to use corect menu item ID http://screencast.com/t/alHjJaJoG89 , your ID will be different
4. Edit theme folder / js / scripts.js file and paste custom js code below at the end of file
type: 'iframe' // this overrides default type
Remember to replace ID inside the code with your ID. Also you can search for some plugin which can enqueue java script and paste the code inside that plugin. Also remember to replace src for vimeo video with your vimeo url.
5. Last thing is to add custom css code in order to fix z-index for vimeo popup window. You can apply custom css code with simple custom css plugin.
Finally result should look like this - http://screencast.com/t/gTcTyh0oi
Please note that for this kind of edits you should consider using child theme, create file structure like it is in main theme and make edits from there. In this way your changes will be permanent after theme updates.