【Objective】Based on the multi- modality radiomics which combined baseline transrectal ultrasound （ERUS），shear wave elastography（SWE）and multi-detector row CT（MDCT），a predictive model of lymph node metastasis in rectal cancer was established.【Methods】From April 2015 to November 2016，we collected 93 patients with rectal cancers in this study. All patients underwent ERUS，SWE examination，and MDCT scans. Radiomics parameters of all images were analyzed. All the cases received radical resection and pathological examination. Statistical analysis was performed using R language，and the US score and CT score were calculated for each patient by a weighted linear combination equation. Combined with the clinical and laboratory indexes，US score and CT score，we established a predictive model，and compared with the conventional imaging standard.【Results】The regression results showed that the risk factors of US were as follows：lymph node size，the proportion of the tumor occupied lumen circumferential，uT staging，contrast of tumor stiffness，maximum modulus in lymph node，and stiffness standard deviation of lymph node. Among the MDCT index，the number of lymph nodes，minor axis of lymph node，and the minimum CT value of lymph node in the venous phase were predictive factors for lymph node metastasis. Combined with the 5 indicators of gender， age，CEA，US score and CT score，we established a nomogram prediction model. The C-index of nomogram was 0.878. The calibration curve showed that the predicted effect of nomogram had good agreement with the actual situation of lymph node metastasis. Compared with the lymph node length as traditional diagnostic criteria（US = 0.691，CT = 0.749），the predictive model of C-index was significantly improved（0.79 for US，0.83 for CT）.【Conclusion】The predictive model of multi- modality radiomics（ERUS，SWE and MDCT） could assess the biological characteristics of rectal cancer and lymph nodes. Compared with traditional imaging diagnostic criteria，the model improved the predictive accuracy of lymph node metastasis.