We present a new kind of dual-hole unit-based porous-core hexagonal photonic crystal fiber (H-PCF) with low loss and high birefringence in terahertz regime. The proposed fiber offers simultaneously high birefringence and low effective material loss (EML) in the frequency range of 0.5-0.85 THz with single-mode operation. An air-hole pair is used inside the core instead of elliptical shaped air holes to introduce asymmetry for attaining high birefringence; where the birefringence can be enhanced by rotating the dual-hole unit axis of orientation. The proposed H-PCF provides a birefringence of ~0.033 and an EML of 0.43 dB/cm at an operating frequency of 0.85 THz.