No, you can drop the direct object "le" and "it", respectively, but you incur a change in meaning both in French as well as in English. "You asked me." is something slightly different from "You asked me about it."
Tu me l'as demandé. -- You asked me about it. (That sounds better than just "it".)
Te m'as demandé. -- You asked me.
-- Chris (not a native speaker).