Asphalt concrete (AC) is a granular composite material stabilized by the presence of asphalt binder. In continuum modeling for AC, the macroscopic response has been lacking the ability to explicitly account for the effect of the microstructure distribution. In this study, an elasto?viscoplastic continuum model is developed to predict AC response and performance. The model incorporates a Drucker?Prager yield surface modified to capture the influence of stress path direction on the material response. Parameters that reflect the directional distribution of aggregates and damage density in the microstructure are included in the model. The elasto?viscoplastic model is converted into a numerical formulation and implemented in finite element (FE) using user?defined material subroutine (UMAT). A fully implicit algorithm in time?step control is used to enhance the efficiency in the FE analysis. The FE model is used in this study to simulate experimental measurements with three mixes design under different confining pressures and strain rates.