What is the value of segment AB in the quadrilateral of the question below?

Using your notation,

1. Angle Bisector Theorem on $\triangle AHD$ yields $$\overline{AH} = \frac{12}7 \overline{HC}.$$
2. Note that $BC$ bisects $\angle HCA$. Angle Bisector Theorem on $\triangle HCA$ gives, together with 1. $$\overline{AC} = \frac{\overline{AB}\cdot \overline{HC}}{\frac{12}7\overline{HC}-\overline{AB}}.$$
3. Menelaus's Theorem on $\triangle HCA$ with secant $BD$ yields $$(\overline{AC}-3) \cdot 7 \cdot (\overline{AH}-\overline{AB}) = 3\cdot (\overline{HC} + 7) \cdot \overline{AB}.$$Use now 1. and 2. in the previous equation to get $$7\left(\frac{\overline{AB}\cdot \overline{HC}}{\frac{12}7\overline{HC}-\overline{AB}}-3\right)\left(\frac{12}7\overline{HC}-\overline{AB}\right)=3\cdot (\overline{HC} + 7) \cdot \overline{AB}$$which is equivalent to $$\require{cancel} 7\overline{AB}\cdot \overline{HC}-36\overline{HC}+\cancel{21\overline{AB}}=3\overline{AB}\cdot \overline{HC}+\cancel{21\overline{AB}}. $$ Simplifying and dividing by $\overline{HC}$ (which is not $0$) leads to the result.