matrix – FindInstance won’t compute this simple expression

A slick way is to compute the Cholesky decomposition of the starting matrix, and then impose conditions on the diagonal of the resulting upper triangular matrix:

Diagonal(CholeskyDecomposition({{a, f, g}, {f, b, h}, {g, h, c}}))^2 /.
Conjugate -> Identity
   {a, b - f^2/a, c - g^2/a - (-((f g)/a) + h)^2/(b - f^2/a)}

Reduce(Thread(% > 0), {a, b, c, f, g, h})
   a > 0 && b > 0 && c > 0 && -Sqrt(a b) < f < Sqrt(a b) &&
   -Sqrt(a c) < g < Sqrt(a c) &&
   (f g)/a - Sqrt((a^2 b c - a c f^2 - a b g^2 + f^2 g^2)/a^2) < h <
   (f g)/a + Sqrt((a^2 b c - a c f^2 - a b g^2 + f^2 g^2)/a^2)

(* find 10 instances *)
FindInstance(%, {a, b, c, f, g, h}, Integers, 10)
   {{a -> 89, b -> 48, c -> 49, f -> 9, g -> 21, h -> 21},
    {a -> 134, b -> 59, c -> 5, f -> -37, g -> 20, h -> -6},
    {a -> 530, b -> 8, c -> 72, f -> 16, g -> -176, h -> -7},
    {a -> 532, b -> 49, c -> 10, f -> -153, g -> 23, h -> -5},
    {a -> 638, b -> 89, c -> 11, f -> -209, g -> -44, h -> 9},
    {a -> 642, b -> 38, c -> 78, f -> -57, g -> -162, h -> 14},
    {a -> 663, b -> 89, c -> 28, f -> -220, g -> -83, h -> 15},
    {a -> 769, b -> 62, c -> 24, f -> -145, g -> -73, h -> 34},
    {a -> 816, b -> 55, c -> 12, f -> -193, g -> -15, h -> -4},
    {a -> 898, b -> 49, c -> 93, f -> -125, g -> -191, h -> -9}}

I’ll leave you to figure out how to impose your extra condition of the determinant to be equal to $3$.