Genesis 40:3-4

40:3 so he imprisoned them in the house of the captain of the guard in the same facility where Joseph was confined. 40:4 The captain of the guard appointed Joseph to be their attendant, and he served them.

They spent some time in custody.

Genesis 41:10

41:10 Pharaoh was enraged with his servants, and he put me in prison in the house of the captain of the guards – me and the chief baker.