Closing a form and then call another one

Change

this.Close();

To:

this.Hide();

Because you can’t Close Main Application window and want to application runs after it.
You must hide main form or change main window to window who was still opened.

In this case you must close main window after ShowDialog() was ended.
Then you must add on the end of this button event function this.Close()

Your code new code is:

private void buttonStartQuiz_Click(object sender, EventArgs e)
    {
        // hide main form
        this.Hide();

        // show other form
        Form2 form2 = new Form2();
        form2.ShowDialog();

        // close application
        this.Close();
    }