Steep black tea in hot water, along with brown sugar. Let sit for about 10 minutes. Remove the tea bags and cool completely in the fridge.
Meanwhile, bring a pot of water to boil. Add the tapioca pearls and boil until softened, about 20 minutes. Use a strainer to drain the pearls. Return the pearls into an empty pot.
Add brown sugar and hot water to the pot. Soak the pearls in brown sugar mixture for 30 minutes.
Divide the pearls among the popsicle molds. Add milk tea until the mold is filled. Insert the popsicle sticks, and then freeze for at least 6 hours, or overnight.