Cells usually have a polarized shape in directional cell migration. This cell polarity may result from external cues, such as a gradient of chemo-attractants (chemotaxis), or a gradient of mechanical properties of substrate (durotaxis), and it can also arise from internal cues so that the cells self-polarize spontaneously and maintain the polar motile state for a long time. However, the mechanisms that control cell polarization have not been fully understood, and particularly, the relationship between the polarized shape and cell migration behaviors is not yet clear. In this study, we propose an energy model to study the cell polarization energy by considering the effect of matrix rigidity, cell shape, and organization of the cytoskeleton. We then propose a parameter called “motility factor” for depicting the relationship between the cell shape and the driving force of cell migration. We demonstrate that the fibroblast-like cell shape and keratocyte-like shape both have an optimal polarization angle corresponding to the most stable cell shape. Fibroblast-like cell shape also has an optimal tail length of the polarization. Furthermore, we find that the cell free energy biphasically depends on the matrix rigidity, i.e. that there is an optimum matrix rigidity for the most stable shape. And the motility factor also biphasically depends on the matrix rigidity, but the trends of the dependence are opposite to that of cell's free energy, which implies an optimum matrix rigidity for the highest speed. The optimum matrix rigidity for the most stable cell shape and that for the highest cell speed are consistent, suggesting that the most stable cell shape is favorable to the fastest cell migration. This study provides important insights into the relationship between cell polarization shape and cell migration behaviors.