Wardiansyah was among dozens of motorcycle taxi drivers who decided to take action in the city of Padang, reports BBC.
Phone footage shows the convoy leaving the hospital, with the tiny body of Alif Putr cradled in one man's arms.In Islam burials should normally take place as soon as possible.
However, the family was told by M Djamil Hospital that they couldn't take six-month-old Alif to be buried until they had paid their bills.
It was then the taxi drivers decided to act, as the boy's uncle was one of their own.
"We took action after finding out the family couldn't take their child for burial because they couldn't pay the 25 million rupiahs ($1,774; £1,370) in bills they owed," Wardiansyah explained to BBC Indonesia.
"The security officers tried to stop us, but they gave up because there were so many of us."Video of the incident went viral in Indonesia and sparked a renewed debate about the treatment of parents who can't afford to pay their medical bills.
There have been numerous cases in the past of hospitals and clinics in Indonesia holding newborn babies hostage until payments for medical bills are paid.
Under President Joko Widodo, a universal healthcare program has been rolled out across the country, but the scheme has been beset with funding problems and many poor families have failed to register.
The baby's mother, Dewi Surya, said the family was in the process of signing up for the healthcare scheme when Alif got sick.
He underwent surgery but died on Tuesday morning.
"The hospital wanted us to pay the bill; the administration process was being dragged out," she said. "The drivers were angry. So they took Alif by force."
"Poor Alif was waiting too long in the morgue," she added, speaking through tears at his funeral.
The hospital later apologised and promised that it wouldn't happen again.
Director Yusirwan Yusuf said the the outstanding bills had now been covered by the hospital board, and described the incident as a misunderstanding.
"The hospital only found out about the family's condition when they filed a complaint with our official," he said. "We are a public hospital and we never even ask patients whether they have money to pay for treatment."
However, Mr Yusuf criticised the raid by the bikers, which he called reckless and dangerous.
"We have a standard operating procedure and it was broken. That's outrageous. What if the body had a contagious disease? Who will be held responsible?"
A representative of the motorcycle taxi drivers also later apologised to the hospital management.
"I, on behalf of my colleagues, apologised for the incident and we aim to restore the good name of the hospital. We didn't know the procedure. It was taking too long, that's why we took that decision," said driver Alfiandri.