What does $F(\varphi)(z)$ mean?
In "The Riemann boundary value problem with respect to the perturbation of boundary curve", p.832
The Cauchy type integral
$$F(\varphi)(z)=\frac{1}{2\pi i}\int_L \frac{\varphi(t)}{t-z}\,dt$$
Why are there two parentheses next to each other? Does it mean $F$ is a function of $\varphi$ and $z$? If yes, what is the difference between $F(\varphi, z)$ and $F(\varphi)(z)$?
$F$ is a function in $\phi$ that returns as value a function in $z$. A more visible way to write this would be $$ \left( F(\phi)\right)(z)=\ldots$$ or $$F(\phi)=z\mapsto \ldots$$ or maybe $$ F_\phi(z)=\ldots$$
These types of notations are fairly common in Differential geometry of manifolds, specially on Vector fields.
The symbol $\displaystyle F(\varphi)(z)$ means $F$ is a mapping from the space $\mathcal C\times \mathbb C$, where $\mathcal C$ is the set of (suppose) all continuous functions over $\mathbb C$, to $\mathbb C$. So $\displaystyle F:\mathcal C\times \mathbb C\to \mathbb C$ defined by, $$F(\varphi)(z)=\frac{1}{2\pi i}\int_L\frac{\varphi(t)}{t-z}dt.$$ So for each fixed $\varphi\in\mathcal C$ $F(\varphi)$ is actually a map from $\mathbb C\to \mathbb C$.
Yes indeed $F$ is a function of $\varphi$ only but to evaluate this we need $z$.
Think like $\displaystyle \frac{\partial}{\partial x}$ this is a function from certain function space to some function space but we need an $x$ to evaluate this. In Differential geometry we often write $$\frac{\partial f}{\partial x}(x)=\frac{\partial f(x)}{\partial x}=\frac{\partial}{\partial x}(f)(x).$$
Now can you see the similarity?
Answer to your last question, there is no difference between $F(\varphi,z)$ and $F(\varphi)(z)$, but conventionally we write the second to maintain the similarity with Vector Fields. We can also write $F(\varphi(z))$.