Depression is a very common mental health disorder with a devastating social and economic impact. It can be costly and difficult to detect, traditionally requiring a significant number of hours by a trained psychiatrist. Recently, machine learning models have been trained for depression screening using patient voice recordings from an interview driven by a virtual agent. To engage the patient in a conversation and increase the quantity of responses, the virtual interviewer asks a series of follow-up questions. For obvious reason, a subject would prefer to have to answer fewer questions. Unfortunately, it is unknown to date if these series of follow-on questions have a tangible impact on the performance of deep learning models for depression classification. Therefore, we study the effect of including one, two, or more follow-up questions on depression screening. We apply a pre-trained-on-voice transfer learning model, namely, VGGish-based model, to classify different subsequences of audio clips. We find that follow-up questions can help to increase the F1 score for the majority of questions, with two questions resulting in the highest F1 scores. Our results can be leveraged for the design of future mental illness screening applications by informing us not only about the selection of the most effective questions but also the number of follow-up questions typically required for screening to produce reliable results.