This would mean. The "matter" would be a situation or incident. "Treating it lightly" means; not putting too much importance into it. "Where as i myself was never more serious" means YOU or the person saying the sentence is much more serious about the incident, the situation/incident is much more important to he/she (speaker).

你对待这个问题挺随便，但是我也不比你认真。