Go into calendar settings and select Validation and put in the following code to validate the time between the (Start Time) and (End Time).
Example where the start time is >= 8am and the end time is <= 4pm.
=AND(TIME(HOUR((Start Time)),Minute((Start Time)),Second((Start Time)))>=TIME(8,0,0),TIME(HOUR((End Time)),Minute((End Time)),Second((End Time)))<=TIME(16,0,0))
Then add the following message in the validation user message box: Please select time between 8:00 AM and 4:00 PM. All Day Events are not allowed.
To expand upon the numbered section of the code we have (8,0,0) standing for (8:00 AM) in (hours, minutes, sections) using the 24 hour clock numbering system. Feel free to adjust the time as needed. I threw in the note about all day events not allowed because it removes time and causes the validation to become false when no time is present in the start/end fields.
I found the answer on this forum and modified the code. https://social.technet.microsoft.com/Forums/sharepoint/en-US/32e9e3c4-2d60-41e2-87f6-f9ae14b6977c/how-to-create-a-validation-rule-for-a-specific-time-interval?forum=sharepointadminprevious