Minus responsibility there is no leadership.
Whosoever accepts responsibility of the well being of others has assumed the leadership role over their lives.
The people you serve are the people that will willingly submit to your leadership.
Responsibility for the well being of others is all about thinking of them in your mind and bearing their burden on your shoulder.
People will naturally submit to your authority when they see that you are truly willing to serve them.
Keep serving if you must keep leading.