Can I add a not null column without DEFAULT value

No, you can't.

Because if you could, SQL wouldn't know what to put as value in the already existing records. If you didn't have any records in the table it would work without issues.

The simplest way to do this is create the column with a default and then remove the default.

ALTER TABLE dbo.MyTable ADD
MyColumn text NOT NULL CONSTRAINT DF_MyTable_MyColumn DEFAULT 'defaultValue'
ALTER TABLE dbo.MyTable
DROP CONSTRAINT DF_MyTable_MyColumn

Another alternative would be to add the column without the constraint, fill the values for all cells and add the constraint.

Add the column to the table, update the existing rows so none of them are null, and then add a "not null" constraint.

No - SQL Server quite reasonably rejects this, because it wouldn't know what value existing rows should have

It's easy to create a DEFAULT at the same time, and then immediately drop it.

I use this approach to insert NOT NULL column without default value

ALTER TABLE [Table] ADD [Column] INT NULL
GO
UPDATE [Table] SET [Column] = <default_value>
ALTER TABLE [Table] ALTER COLUMN [Column] INT NOT NULL

No.

Just use empty string '' (in case of character type) or 0 (if numeric), etc as DEFAULT value