In order to do the (suspected) damage reflected on your initial ALT, the virus
would be certainly pass the incubation period
, and the antigen would be detected in your blood. In addition, if your baby is infected by HepB during delivery, the virus normally stays in the immunotolerant stage for many years where your baby immature immune system just ignores the virus, which result in normal ALT. From what I know of HepB, it doesn't fit your case.