Goku more or less abandoning a baby to his captors (albeit temporarily) is callous even for him, but Goku’s almost always had a good reason for his more controversial decisions. Why give Cell a Senzu Bean without telling Gohan? Because he knows his son needed that push and he was right. That same logic applies in wanting to go after the Tamagami before Dende. Goku may be in it for the thrill of the fight, but he also understands how much weaker he’s gotten in the Demon Realm. It’s not a coincidence that every episode since Episode 2 has featured at least one major scene where Goku comments on how much weaker he is now.
This is important, as it establishes that Goku knows he’s fighting at a major disadvantage now. This goes double with Goku pointing out how dangerous the Demon Realm is in Episode 4. Not just that, Goku suggests that Glorio could actually be stronger than the Supreme Kai early on in the episode. Shin is said to be strong enough to kill Frieza in one shot in Dragon Ball Z, which would mean Demon Realm monsters could have a Power Level ranging anywhere from 530,000 to over 1,000,000. All this to say, Dende is going to stay in danger until Goku can master his abilities and grow stronger in his mini body.
Keep in mind that Goku is a veteran martial artist by Dragon Ball DAIMA. He’s been doing this for decades and is hot off the Majin Buu saga where he acted as a master to both Goten and Trunks. He knows his limits, he knows his potential, and he knows what he’s capable of. After a lifetime of training his body, Goku understands he has to go through the process again in such a weakened state. By focusing on the Tamagami first, Goku will grow stronger and unlock his old skills by pushing himself – he might even qualify for Zenkai boosts again. We know he will turn Super Saiyan as seen in the opening credits, but it’s unclear if he currently can.
It’s possible Goku needs to be pushed to his limits again before being able to trigger SSJ again. Couple all this with the mention of stamina throughout the episode and it seems clear that Goku is using the Dragon Ball hunt as a means of growing more powerful. If they head to Dende too early, there’s a real chance they’d die trying to save him. Instead, facing the Tamagami gives Goku an opportunity to get used to his body and the Demon Realm’s bad air. Goku obviously shouldn’t be so excited at the prospect of going on an adventure over saving a baby in active danger, but his logic is sound and it was honestly sounder when his original plan was using the Dragon Balls to regain their bodies.