how to do a simple scaling animation and why isn't this working?

i just read in stackoverflow i can only concatenate animation with delay, so i tried this here which simply shrinks and then scales the circle again. unfortunately the shrinking doesn't work!? if i comment out the growing, shrinking works...

struct ContentView: View {
    
    @State var scaleImage : CGFloat = 1
    
    var body: some View {
        VStack {
            Button(action: {
                withAnimation(Animation.easeInOut(duration: 1)) {
                    self.scaleImage = 0.01
                }
                
                withAnimation(Animation.easeInOut(duration: 1).delay(1.0)) {
                    self.scaleImage = 1
                }
            }) {
                Text ("Start animation")
            }
            Image(systemName: "circle.fill")
                .scaleEffect(scaleImage)
        }
    }
}

Here is possible approach (based on AnimatableModifier). Actually it demonstrates how current animation end can be detected, and performed something - in this case, for your scaling scenario, just initiate reversing.

Tested with Xcode 11.4 / iOS 13.4

Simplified & modified your example

struct TestReversingScaleAnimation: View {

    @State var scaleImage : CGFloat = 1

    var body: some View {
        VStack {
            Button("Start animation") {
                self.scaleImage = 0.01       // initiate animation
            }

            Image(systemName: "circle.fill")
                .modifier(ReversingScale(to: scaleImage) {
                    self.scaleImage = 1      // reverse set
                })
                .animation(.default)         // now can be implicit
        }
    }
}

Actually, show-maker here... important comments inline.

struct ReversingScale: AnimatableModifier {
    var value: CGFloat

    private var target: CGFloat
    private var onEnded: () -> ()

    init(to value: CGFloat, onEnded: @escaping () -> () = {}) {
        self.target = value
        self.value = value
        self.onEnded = onEnded // << callback
    }

    var animatableData: CGFloat {
        get { value }
        set { value = newValue
            // newValue here is interpolating by engine, so changing
            // from previous to initially set, so when they got equal
            // animation ended
            if newValue == target {
                onEnded()
            }
        }
    }

    func body(content: Content) -> some View {
        content.scaleEffect(value)
    }
}