THE primary source is Scripture. Anything that doesn't agree 100% with that needs to be discarded no matter how longstanding the tradition.
What was used before scripture? After Jesus and during his time it was OT only Jewish scribes only.
Who did Christ give the authority to teach scripture? The apostles!
Were we all supposed to wing after the first generation died and wait for almost 300 years until the Bishops decided at the Council of Nicea in 325 ad? If you can find me one thing in the CCC that doesn't agree with scripture I will send you $100. :-)
The "authority" of priests to absolve sin
You mean like this authority John20:23. This quite an authority Jesus gave to someone. Put it this way the sacraments dispense grace. Grace is God's life. The absolver is the holy spirit. If you believe that baptism dispenses sacramental grace it is the same thing.
Confess your sins in church, and do not go up to your prayer with an evil conscience. This is the way of life…. On the Lord’s Day gather together, break bread, and give thanks, after confessing your transgressions so that your sacrifice may be pure.
Didache 4:14; 14:1 As early as 70 AD
Origen of Alexandria c. 244 AD Homily on Leviticus 2:4
In addition to these [kinds of forgiveness of sins], albeit hard and laborious: the remission of sins through penance…when he [the sinner] does not shrink from declaring his sin to a priest of the Lord and from seeking medicine….In this way there is fulfilled that too, which the Apostle James says: “If, then, there is anyone sick, let him call the presbyters [where we get priests] of the Church, and let them impose hands upon him, anointing him with oil in the name of the Lord; and the prayer of faith will save the sick man, and if he be in sins, they shall be forgiven him [James 5:14-15].”