Validation using Validating event and ErrorProvider - Show Error Summary

You should first correct your validating events this way:

private void textBox1_Validating(object sender, CancelEventArgs e)
{
    Regex regex1 = new Regex(@"^[a-zA-Z]+$");
    if (!regex1.IsMatch(textBox1.Text))
    {
        //To set validation error
        errorProvider1.SetError(textBox1, "Nosaukums nedrīskt saturēt ciparus!");
        //To say the state of control in invalid
        e.Cancel = true;
    }
    else
    {
        //To clear the validation error
        this.errorProvider1.SetError(this.textBox1, "");
    }
}

Then you should use ValidateChildren method to check if there is a validation error or not, then you can get a list of all errors and show to user this way:

private void button1_Click(object sender, EventArgs e)
{
    if (this.ValidateChildren())
    {
        //Here the form is in valid state
        //Do what you need when the form is valid
    }
    else
    {
        var listOfErrors = this.errorProvider1.ContainerControl.Controls.Cast<Control>()
                               .Select(c => this.errorProvider1.GetError(c))
                               .Where(s => !string.IsNullOrEmpty(s))
                               .ToList();
        MessageBox.Show("Please correct validation errors:\n - " +
            string.Join("\n - ", listOfErrors.ToArray()),
            "Error",  
            MessageBoxButtons.OK, MessageBoxIcon.Error);
    }
}

A Sample screenshot:

Note:

• You should not use Clear method of error provider to set valid state to control, you should use SetError, for example this.errorProvider1.SetError(textBox2, "");
• You should call e.Cancel=true when there is a validation error.
• In sample codes I assume that all your controls including the error provider placed directly on your form and not in a container control.
• I also recommend to change validation behavior of form by setting AutoValidate property of form to EnableAllowFocusChange in design time or by code in Load event of form this way:

To change validation behavior of form:

this.AutoValidate = System.Windows.Forms.AutoValidate.EnableAllowFocusChange;