Genesis 40:3-4

40:3 so he imprisoned them in the house of the captain of the guard in the same facility where Joseph was confined. 40:4 The captain of the guard appointed Joseph to be their attendant, and he served them.

They spent some time in custody.