Women are generally smaller than men, but even if you have men and women of equal height and build, the man will generally be stronger than the woman due to "biological differences in muscle mass and hormone levels, with the strength disparity being most noticable in upper body strength; this is generally true even when comparing individuals with similar body composition."
From a biological standpoint, the gender virtually always matters.