Network measurement and management are critical to the future smart network QoS and enhancing user QoE. Accurate prediction of network status can support network measurement and provide extra time for network management. However, it is challenging to achieve high accuracy in real-time due to the complexity of user behavior as well as the diversity of network applications. The future SDN will enable data-driven techniques to provide accurate network traffic prediction from access gateways. To demonstrate the concept, a DL-based encrypted packet classifier is first developed in this work to identify network applications. With the achievement of application awareness, a DL-based network traffic prediction scheme is further proposed and developed to provide accurate network traffic prediction. Datasets of network packets from an open-source as well as traffic flow collected in real life are applied to conduct evaluations and case studies. The evaluation results demonstrate that the proposed network traffic classification and prediction framework can successfully predict network traffic in access networks. The proposed framework will contribute significantly to the transition toward intelligent communication and networking to enhance user QoE.