for me starting to add some kanjis early, helped me a lot with learning new vocabulary so if you are confident in your hiragana and katakana I think it wouldn't hurt
Personally, I started including 漢字 in my learning schedules when I was confidently writing and reading kana, mainly hiragana and to a lesser extend katakana. I feel you will find kanji much easier to learn and remember when able to read and write kana, as you will be able to read furigana (the small written kana above unknown kanji in the learning stage)
