It sounds as if you have a calendar list, and the issue is clicking the “Add” button launches the list’s default out of the box content type New form, not your custom form.
When you refer to your custom form, my assumption is that you are referring to a custom event content type that can be added to the calendar list. If that is the case, 1] add your custom content type to the list, 2] and then modify the “Change New Button Order” settings and position the custom content type as the first available option. The default action for a calendar’s “Add” button is to open the content type in the first position.
If you are referring to a form that is not associated with an event content type, are you able to modify the calendar’s default New form by 1] hiding the list web part and 2] adding a link to your custom form? However, this would require users to click twice to get to your form, which isn’t ideal. Another option would be to point users to a web page that displays the calendar, versus accessing the calendar itself. On the web page you can include a prominent link to your custom form. If a user’s eye is drawn to the link text for the custom form, they may be less inclined to click the “Add” button in the calendar.