I like having the sidebar showing in the new Photos app (View -> Show Sidebar). But using View -> Sort -> Sort by Name only sorts the content on the main content panel to the right, not the sidebar.
To sort the Albums list on the left by name, go to View -> Albums (Cmd + 3), then go to View (again) -> Sort -> Sort by Name.
Because you've chosen a view where the albums are on the main panel, the instruction to sort them by name will work, and will also sort the left sidebar column by name.
This option does not auto-update, so it needs to be done every now and then to keep the list sorted.